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ABSTRACT 

A Study was conducted to develop a model for a 
computer-based career information delivery system for use in New York 
State occupational education programs. The study included a 
literature review and five major substudies (information validation, 
user satisfaction, and vendor surveys; site visits to selected 
schools and agencies using career information systems; and a 
demonstration of career information delivery systems to a system 
review panel). The literature review yielded a list of 85 information 
components that was confirmed by the validation survey and enlarged 
by surveys of principals, counselors, students, and parents. Of the 
five career information systems presented by vendors, four were i.ated 
satisfactory and just passing. It was decided that the state should 
pursue the development and utilization of career information delivery 
systems. No single system should be adopted; however, all systems 
considered for use should undergo some sort of approval process. 
Activities such as staff development workshops and printed 
conanunications should be developed as soon as possible, and a career 
information delivery system coordinating office should be created to 
oversee the coordination and continued improvement of career 
information systems for vocational students. (Appendixes include 
copies of all the project surveys and related materials, 
correspondence, and forms.) (MN) 
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EVALUATING COMPUTERIZED CAREER INFORMATION SYSTEMS 
FOR USE WITH OCCUPATIONAL EDUCATION CURRICULA 

EXECUTIVE SUMMARY 

The overall purpose of the New York State study, "Evaluating 
Computerized Career Information Systems for Use With Occupational 
Education Curricula" was to identify a New York State model for a 
computer-based career information delivery system or systems to 
meet the objectives of occupational educfition curricula and 
implementation of guidance plans, as well as the needs of junior 
high school students, high school students, and adults, including 
special needs populations in those groups. 

After the review of the literature, five major activities or 
sub-studies were planned and carried out. They were: 

1. The Information Validation Survey . Information 
components and design considerations which were identified 
through the review of the literature were presented in a 
survey instrument with a Likert-type scale ranging from 
"highly desirable" to "highly undesirable." These 
instruments were sent to a national panel of experts in 
career guidance and a state panel of experts in guidance and 
occupational education. 

2. The Surveys of User Satisfaction . Surveys to determine 
the degree; of satisfaction with systems in use as well as 
the overall needs and uses for a system were constructed for 
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principals, counselors, students and parents. A selection 
of schools throughout the State using C-LECT, CHOICES, CIS, 
Discover, GIS, and SIGI were surveyed. 

3. The site visits . Selected schools and agencies using the 
major systems were visited to obtain information about the 
uses of and satisfaction with systems through in-depth 
interviews with counselors at those sites. 

4. Request for Information from vendors . Based on the 
results of the Information Validation Survey, the Surveys of 
User Satisfaction, and the site visits, a request for 
information regarding system components and costs was 
developed and sent to the vendors identified as active in 
New York State. 

5 . The demonstration of career information delivery systems 
to the System Review Panel . The same vendors were invited 
to make a structured presentation to a panel of guidance, 
occupational education, rehabilitation, and labor 
professionals both from State government and local agencies 
and schools. A rating instrument was designed with a 
Likert-type scale for the System Review Panel's evaluation 
of the systems. 

The review of the literature yielded a list of 85 
information components and design considerations which were 
validated through the responses of the national and state panels 
of experts to the Information Validation Survey. 

Surveys of User Satisfaction of principals, counselors, 
students, and parents confirmed and enlarged the list. In 
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general, these surveys showed that the CIDS were being used in 
the schools primarily for guidance with some classroom and group 
instruction activities. The students' responses showed that they 
found the systems easy and enjoyable to use and more useful for 
information gathering than for career decision making, thus, 
indirectly, supporting the idea that a career information 
delivery system is a resource to be used as part of an overall 
career guidance or education program. The parents' responses 
showed that they strongly supported the use of computerized 
systems in their schools and the use of computers for career 
information delivery and exploration, but not for values 
clarification. Counselors were highly supportive of computerized 
career information, but only moderately satisfied with the 
systems they had. Counselors expressed particular concerns with 
the quality and timeliness of the information in the systems and 
the adequacy of the training. 

Five major vendors of CIDS in New York State responded to 
lengthy requests for information, provided printouts of 
information which was analyzed for comprehensiveness and 
accuracy, and demonstrated their systems to a review panel. A 
process for rating the CIDS taking all of this into consideration 
was developed and used for measuring the part and total value of 
each system. While different systems excelled or were weak in 
different aspects — information components contained, system 
design considerations, user support materials, information 
comprehensiveness, rating by the System Review Panel ~ the final 
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analysis shewed that four of the systems were satisfactory (Ii; descending 
Order : CIS, GIS, CHOICES and Discover) and one (C-LBCT) just passing, rais 
oonfirned tlie counselors' levels of satisfaction, or dissatisfaction, with 
the CIDS in use. 

While costs were not analyzed as part of the total rating systan, they 
were gathered through the RFI and cortparative analyses presented. This 
analysis shewed large variations in basic system delivery, developnental, 
and total costs. It also revealed iiqportant differences among the CIDS in 
st\ch issues as entry and ownership of state developed data and files. 

The recrir ro ^ nd a t iffRs >idc^ made took into consideration a nutter of 
factors. First, there was a high degree of general support for carputerized 
career information delivery systenis expressed by the various user cortmun- 
ities. Second, although there vsere weaknesses identified in the systems, 
there was generally more satisfaction than dissatisfaction, and there has 
been a varying degree of investitent of resources in each of the CIDS by 
schools and agencies in the State. Third, the greatest area of dissatis- 
faction could be interpreted as a hoe aiid cry for more accurate, more 
t?>-to-date, and Bcore specific occt^sational and. educational inforxtation. 
Fourth, an analogy to liie tradition in York State for approved, rather 
than adopted, texts may be drawn to educational software, and thence to 
CIDS. 

Eight recornnendations were made. 

1. The State should pursue the development and utilization of career 
information delivery systems. 

2. No single system should be adopted. Instead, systems should be rated, 
and if satisfectory, ^proved. 

6 
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3. Sane neaning should be ascribed to Jie fact that a system is approved. 

4. The five systems rated for the study (C-LBCT, C310ICES, CIS, Discover, 
and GIS) shotild be approved. (SIGI, vrfiich did not carplete the process, and 
any other systems should not be considered approved until they are rated.) 

5. Uie process for system approval and the related rating sheets presented 
in the study should be adopted and utilized as the process for approving 
systems. 

6. Activities such as staff development workshops and printed ccmnunica- 
tions should be undertalasn as soon as possible to inform those responsible 
for buying and using CUDS of the rvasults of this study. 

7. The State of New York should enhance its primary data swaries data 
collection so that the series can incorporate the information cot^onents and 
level of detail needed by CIDS as indicated by the information x^^iir!a^-inn 
survey and surveys of user satisfaction. Since the Department of labor has 
primary responsibility for such activities, 'staff and resources should be 
made available as needed to sugpo rL this reocnraendation. 

8» The f unc ti on of coordinating and continuously iii|)roving career inforaia- 
tion delivery should be tbcused in one location. A tmit, the ra-nagir 
Infonnation Delivery System Obordinatijig. Office (CIDSOO) should be created 
Irt the State Bdiication TSffpart^ netwock of users already 

exists. In order to cooi'ndinate and inprove career information delivery, 
this unit should: 

A. develop state career infonnation ~ inclioding information about 
occcpationSi^ secondary areas of concentration; postsecondary educational 
programs; colleges and universities; proprietary schools; financial aids and 
scholarships. 
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B. be responsible for seeing that the state information is incorpor- 
ated in the approved CIDS, with the State carrying out the data entry> 
provide oarputer expertise in CIDS management and developnent, 

D. be responsible for rating and approving other CIDS using the 
process developed in this stu^. 

E. negotiate procurement contracts with all approved CIDS. 

F. disseminate information and conduct staff developnent activities 
regarding CIDS. 

With the establishment of the CIDSCOr New York will be taking the best 
possible approach for serving CIDS users across the State. This approach 
will provide T:5>-to-date, accurate career information, qualify control and 
standards, and meet the locsLL needs and preferences. This innovative 
approach is particularly appropriate to the size, diversity, and resources 
of Ifew York State. 
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INTRODUCTION 



The overall purpose of the New York State study, "Evaluating 
Computerized Career Information Systems for Use With Occupational 
Education Curricula," was to identify a New York State model for 
a computer-based career information delivery system or systems to 
meet: the objectives of occupational education curricula and 
implementation of guidance plans, as well as the needs of junior 
high school students, high school students, and adults, including 
special needs populations in those groups. 

The impetus for the study was the growing awareness of 
career information needs identified in the implementation of the 
Regents Action Plan, particularly in the development of the new 
occupational education curricula, "Home and Career Skills" and 
"Introduction to Technology" mandated for the seventh and eighth 
grades and "Introduction to Occupations," as well as in the 
development and review of guidance plans. 

The study was conducted by the principal and associate 
investigators, Deborah Perlmutter Bloch, Ph.D. and Joyce Ford 
Kinnison, from March 1, 1986 through August 31, 1986. It was 
funded by the New York State Education Department, Office of 
Occupational and Continuing Education, James A. Kadamus, 
Assistant Commissioner, and carried out under the auspices of the 
New York State Occupational Information Coordinating Committee. 
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The statutory members of the New York SOICC are: 
-New York State Education Department 

James A. Kadamus, Chairman of SOICC and Assistant Commissioner 

Office of Occupational and Continuing Education 

-New York State Department of Labor 

Robert Gulotty, Director 

Job Service Division 

-New York State Department of Labor 

John F. Hudacs, Executive Deputy Industrial Commissioner 

-New York State Department of Commerce 

Dennis Rapp, Deputy Commissioner 

Division of Policy and Research 

-New York State Department of Social Services 

Jack Ryan, Director 

Commission for the Blind and Other Visually Disabled 
-New York State Education Department 
Richard Switzer, Deputy Commiysioner 
Office of Vocational Rehabilitation 

Project oversight was carried out by the Project Review 
Committee which consisted of the following: 
-Jeremy Schrauf (Chairman) 
Director of Research and Statistics 
New York Stite Department of Labor 
-Barbara Shay, Chief 

Bureau of Occupational Education Policy Development 
New York State Education Department 
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-Jack Fabozzi, Acting Chief 

Bureau of Career Awareness and Practical Arts Programs 

New York State Education Department 

-Frederick Francis, Director 

Placement, Outreach and Marketing 

Office of Vocational Rehabilitation 

New York State Education Department 

Technical assistance to the project was provided by Robert 
DeFabio, New York State Education Department, and David Nyhan, 
New York State Department of Labor. 

This report is composed of the following chapters: 

Background 

Review of the Literature 

Methodology 

Findings 

Ancillary Issues 

Summary and Recommendations 
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Chapter One 
BACKGROUND 



Introduction 

The overall purpose of the New York State study, "Evaluating 
Computerized Career Information Systems for Use With Occupational 
Education Curricula" was to identify a New York State model for a 
computer-based career information delivery system or systems to 
meet the objectives of occupational education curricula and 
implementation of guidance plans, as well as the needs of junior 
high school students, high school students, and adults, including 
special needs populations in those groups. 

More than a decade of work in developing and using computer- 
based systems for career information has taken place at both the 
national and State levels. This study is set against that 
historical background. Three sections briefly describing the 
background are presented in this chapter. They are: "National 
Activities," "New York State Activities," and "New York State: 
Current Status." 

National Activities 

The movement toward the development and implementation of 
computerized career information systems began in the late 1960's, 
when the U. S. Department of Labor funded a research and 
development project in Oregon (the Occupational Information Access 
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System — OIAS) to design and field test a model labor market 
information delivery system. The computerized system which 
resulted from this seminal work became the Career Information 
System (CIS). Based on the success of the Oregon work, the 
Department of Labor later funded the implementation of 
computerized systfems in eight additional states. 

In the Vocationa] Education Act of 1976 (PL 94-482), the 
Congress mandated the establishment of the National Occupational 
Information Coordinati-ig Committee (NOICC) with State 
Occupational Information Coordinating Committees (SOICC's) to be 
established in the 50 states and seven Trust Territories. 
Growing out of the success of the Oregon CIS for use in career 
guidance and counseling activities and with the advent of an 
interest in promoting the use of computerized systems for 
educational activities across the country, there was included in 
one section of that act, the mandate that the NOICC and SOICC's 
should work to oversee the establishment of computerized 
occupational information systems in each state. Among other 
functions, chese systems were to provide labor market and 
occupational information for students and adults in the process 
of career decision making. 

The mandate for the development of computerized systems for 
career decision-making activities was further expanded in the 
Youth Employment and Training Program Act of 1977 (PL 95-93), the 
CETA Amendments of 1978 (PL 95-524), the Job Training Partnership 
Act of 1982, and the Carl D. Perkins Vocational Education Act of 
1984. 
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Early in 1979, NOICC began a competitive Career Information 
Delivery System (CIDS) grant program to help fund the development 
and implementation of career information systems in the states. 
3y 1986, NOICC, through the CIDS program, had made 26 grants to 
fund career information delivery systems. In addition, the 
program had funded most of the original CIS states to make data 
and delivery system modifications to their systems. 

The major systems which are in operation in the country at 
this time are the Career Information System (CIS), the Guidance 
Information System (CIS), the System for Interactive Guidance and 
Information (SIGI), CHOICES, Discover, C-LECT, and the 
Computerized Occupational Information Network (COIN). The 
delivery modes for these systems include mainframe computer, 
minicomputer, microcomputer, card sort, microfiche, and print, 
with soae sites utilizing combinations of two or more modes. 

In 1984, the Perkins Vocational Education Act introduced a 
new legislative thinking process regarding career guidance 
programs at the state level. Until the passage of this 
legislation, federal laws defined career guidance as a support 
service. The Perkins Act went beyond that definition and 
established state authority for a comprehensive career guidance 
"program" for all youth and adults to meet their needs for career 
planning assistance. 
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New York State Activities 

Efforts to meet the career information needs of the students 
and adults of New York State through the use of computer based 
career information delivery systems have been underway since the 
mid-1970 's. 

Among these early efforts were three studies. The first was 
a pilot program in the New York City high schools and other 
settings conducted by the Institute for Research and Development 
in Occupational Education (Center for Advanced Study in 
Education) at the City University of New York using CIS (Heller 
and Chitayat, 1976.) Following that, in 1930, an in-house 
comparative study of CHOICES and CIS was conducted by the New 
York SOICC. In 1981, a study of adult information needs and 
appropriate systems to meet those needs was completed. It 
evaluated CIS, CIS, and CBEOC, a system then being operated by 
the City University of New York through an Education Opportunity 
Center grant. Each of the studies confirmed the efficacy of Che 
use of computer-based systems and the need for a comprehensive 
state system. Each of the studies identified career information 
needs, both ccjimon occupational and educational needs and the 
unique needs of particular populations. 

None of the previous studies led to the development of a 
comprehensive state system. While they each contained useful 
information on which to build, the current study was broader in 
several key respects. It took into account all currently 
available and appropriate career information delivery systems. 
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It also addressed not only the needs of high school students and 
adults, but also two new major concerns: the needs of junior high 
school students and the methodology associated with successful 
infusion into classroom activities in conjunction with the 
curricula mandated by the Regents Action Plan. 

In addition, individual local education agencies, BOCES, 
colleges, and other agencies have been using some of the major 
career information delivery systems available to them from 
system vendors. In 1979, New York City began to deliver, in 10 
sites, locally developed occupational information and national 
post-secondary educational information, using CIS and CIS in 
combination in its system, MetroGuide. MetroGuide developed to 
the point where, by January 1986, using only CIS software, it was 
delivering locally developed information on occupations, post- 
secondary programs of study, financial aid, and proprietary 
schools and nationally developed college information at the 110 
public high schools and selected other sites in New York City. 
In 1982, MetroGuide developed a sub-system for use in the 
intermediate/ junior high schools to aid in articulation to high 
school. MetroGuide Junior was successfully pilot tested in five 
junior high schools and plans are in progress to utilize it in 
additional schools. 

Currently, while MetroGuide is the only system with a 
central support staff developing local information, the largest 
vendor of national information within the State is GIS. In 
addition to major, widespread use of MetroGuide and GIS, 
Discover, CHOICES, and C-LECT are being used in junior high and 
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high schools throughout the State; SIGI and Discover in 
libraries; SIGI, Discover, and C[:0ICES in colleges; and CHOICES 
i.i vocational rehabilitation offices. At present, although there 
may be some efforts to add state or local information by 
individual districts or systems, there is no centralized or 
organized effort to do this. 

New York State: Current Status 

The background of CIDS development, both in the nation and 
in New York State, pointed to three reasons for the State's being 
interested in looking at a model for a State system. The first 
reason was that many of the schools and agencies in the State 
have no access to a career information delivery system. 
According to estimates made ?.n early 1986, the total number of 
school sites being served by computer-based systems was, at best, 
650. In 1985, according to the State Education Department count 
of public schools, there were 235 middle schools, 353 junior high 
schools, 271 junior/senior high schools, 512 senior high 
schools, and 97 K-12 schools in the State. Fewer than 45% of the 
public schools with junior high school or high school populations 
were being served. In addition, in 1985 there were 2,149 non- 
public schools with junior and senior high school populations as 
well as community colleges, adult learning centers, libraries, 
educational information centers, and a variety of counseling 
agencies, which would benefit from having access to a computer- 
based career information delivery system. 
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The second reason was the potential for inconsistent and/or 
incorrect information being disseminated. In the absence of a 
State model, vendors and local agencies (with the exception of 
MetroGuide) deliver only national information which may or may 
not be correct for New York State. Further, no model exists on 
which to base the selection of a system. With the proliferation 
of microcomputers has come an enormous increase in available 
software which itself varies widely in price and quality. Thus 
school districts, BOCES, and the many other agencies needing CIDS 
may be choosing less than the best system or less than the best 
for the money. 

A third reason *:rose from the inclusion in the Regents 
Action Plan of curricula — "Home and Career Skills", 
"Introduction to Technology", and "Introduction to Occupations" 
(for students who plan a concentration in occupational subjects) 
— which mandate the knowledge and use of career information. 
Without a State model, there existed the possibility of not only 
inconsistency of information and quality of delivery but also 
additional costs for the redundant development, entry, and 
maintenance of the information in local systems. 

The State recognized that the delivery of career information 
involves the coordination of the component activities of a system 
for collecting, providing, and using information for specified 
target audiences. For a career information delivery system to be 
of maximum use to the people it has been purchased to serve, 
there must be a method for insuring: system appropriateness for 
the target population, the accuracy of the data, appropriate 
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vocabulary, an absence of bias and stereotyping, timely data 
update, and the availability of adequate training. These 
activities are best managed at a centralized location. 

Of course, education policy and program planning are bath 
a local and a state function. However, with a coordinated state 
CIDS approach, the quality of the requisite national, state, and 
local information contained in a system can be better assured. 
Further, with the management of career information systems taking 
place at the state rather than the local level, economies of 
scale for costly services, such as information development, 
update, and management and system training can be achieved. 
Further this arrangement allows for coordinated purchasing, 
resulting in the possibility of major dollar savings for local 
school systems. 

In addition to these three factors, the State recognized 
that career decision making in this age of rapidly changing 
technology is not a one-time affair. In school, individuals must 
learn how to make job and educational decisions, how to get the 
information they need for the decision, and how to get the 
information needed to act on the decision. Once out of school, 
people continue to need access to this information. Since it is 
estimated that adults make four to five major career changes in 
their working lives, they need information about skill 
transferability, appropriate jobs to meet their current needs and 
interests, and training and retraining opportunities. In this, 
which has been called the information age, the need for career 
and educational information will continue to grow. The economic 
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haalth of New Ycrk State is linked to the accurate and timely 
provision of that information to its youth, adults, employers, 
educators, and planners. 

With all these reasons as a motivating force, the State 
decided on a course of action to help identify the optimal path 
to take in providing the information needed by all the groups 
cited above. In late 1985, the decision was made to conduct this 
study "Evaluating Computerized Career Information Systems for Use 
With Occupational Education Curricula." 

Given tfie historical background and current status of career 
information delivery systems, the principal and associate 
investigators identified seven factors crucial to completion of 
the study. They were: 

1. the identification (both directly and indirectly through 
a review of the literature) of the career information needs 
related to the implementation of the "Home and Career Skills", 
"Introduction to Technology", "Introduction to Occupations", and 
other occupational curricula; the information needs related to 
the implementation of the State's guidance plans; and the career 
information needs of junior high school students, senior high 
school students, and adults, including populations with special 
needs ; 

2. the validation of those needs by both a state panel of 
experts and a national panel of experts in the field of career 
guidance; 
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3. the examination of users' areas of satisfaction and 
dissatisfaction with systems currently in use within New York 
State, by surveying school administrators, counselors, parents, 
and students and by visiting sites using the systems; 

4. the development of a set of standards for a New Yo k 
State career information delivery system or systems based on the 
assessed needs, the areas of user satisfaction and 
dissatisfaction with systems currently in use, and the 
professional criteria of the field of computer-based career 
information systems; 

5. the evaluation, by a State panel, of the appropriate 
available systems using the criteria developed as a result of the 
above activities; 

6. the evaluation of initial and maintenance, state and 
local costs associated with each of the systems; and 

7. the development of recommendations to the State 
regarding the system or systems to be developed. 



26 



CIDS Evaluation 5 14 



Chapter Two 
REVIEW OF THE LITERATURE 

Introduction 

The major purpose of this review of the literature was to 
Identify a broad base of Information components and computerized 
system design considerations which are necessary and desirable in 
a career information delivery system to meet the needs of the 
citizens of New York State. The specific needs which will be 
addressed are those required: 

-to support three new occupational education programs in the 
State; 

-to implement the new Guidance Guidelines for the State 
which are contained in the new Commissioner's Regulations, 
Section 100.2; and 

-to serve the career Information needs of junior and senior 
high school students, adults, and populations with special 
needs. 

In recent decades there has been an increasing recognition 
of the need for and usefulness of reliable occupational and 
career information for use in occupational instruction, career 
counseling and planning, and job placement activities. This 
recognition by labor market intermediaries and direct 
participants in the labor market led to an Interest in 
identifying needed career information and systematizing it. 
Computerized systems which store, organize, update, and retrieve 

ErIc 27 



CIDS Evaluation, 15 

large quantities of informativ-n for immediate and easy access 
were a logical development (Harris, 1972.) 

Career information is important in career decision making 
to help people become aware of and choose an occupation for which 
they are suited and which they would find rewarding, and to help 
them learn about and understand the range of career and 
edu ».tional opportunities which are available or are likely to be 
available to them (Fry, 1979.) 

When accurate, reliable, up-to-date occupational information 
is provided to people, they have the opportunity to find jobs for 
which they are better suited. With occupational information, 
people learn how to pr pare for specific occupations, the 
abilit.es and skills required in the occupations, and the varying 
working conditions and requirad personal attributes of different 
occupu.tiOwS (Fames, 1975.) Andrisani (1978) pointed out that 
this information could aid the person in selecting an occupation 
for which he or she is suited, as well as aid in forming 
realistic expectations and aspirations about the labor market. 

It is consistent with human capital theory that the 
provision of occupational info mation is an investment which can 
lead to higher earnings. Using a national sample of young men 
aged 14 to 24, Fames and Kohen (1975) found that the youth's 
score on an occupational information test had a significantly 
positive effect on the youth's occupational status and hourly 
wage in the two years following the test. Stevenson (1978) found 
that a measure of labor market knowledge had a positive influence 
on the hourly wages for white and black youth of both sexes. 
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Thus, it can seen that tht' use of occupational information for 
career decision making can havci a variety of economically 
beneficial effects on workers. 

Herr (1981) wrote about the use of a career information 
system for use in career counseling. Although he discussed 
career counseling in general, his main focus was the use of the 
system for in-school students. He stated that, based on the 
research in the field, "There appears to be no doubt that 
students who do not have access to or who do not possess 
occupational information pertinent to their goals are 
disadvantaged in career planning, choice, preparation and 
subsequent occupational success." 

Other studies have shown that access to reliable labor 
market information results in such benefits as: increased market 
competition (Parnes, 1975;) increased productivity (Yavitz and 
Morse, 1973;) and increased equity (Parnes, 1975.) Increased 
availability of information is expected to create an improved 
opportunity to choose an appropriate occupation across 
individuals of all socioeconomic groups. Parnes also suggested 
that an eqtial opportunity to choose an occupation may narrow 
earnings differentials across occupations which are artificially 
inflated as a result of unequal access to the occupations. 

These studies point out the need for and the banefits of the 
use of career information in career decision making. However, 
information must be structured in order to be useful. It is this 
structure which is the delineating characteristic of a 
computerized career information delivery system. 
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An information system may be thought of as an organization 
or network for the collection and/or distribution of 
information. For NOICC/SOICC purposes, the information 
being collected and distributed is related to occupations. 
(TW) purpose, simply stated, is to provide to users the 
occupationally related information necessary for decision 
making (NOICC, 1979.) 

McKinlay and Ross (1978) reported results of the the early 
pilot tests of the computerized Oregon CIS. Some of the 
counselors who were part of the test which provided career 
information via the computer judged the system to be most 
valuable with young, disadvantaged, unmotivated clients. Others 
judged it to be useful for a wide variety of users, including 
bright and articulate college students, slow high school 
students, people with some idea of their goals, people with no 
idea what they want to do, returning servicemen, and potential 
college students. 

Up-to-date, readily available career information can enhance 
the activities associated with vocational and career counseling, 
career decision making, job placement, job search, and teaching. 
Among the individuals identified as needing career planning 
information for these activities are youth, secondary and post- 
secondary students, the unemployed, the incarcerated, new 
entrants and re-entrants to the job market, people with 
handicapping conditions, people making mid-life career changes, 
military returnees, counselors, teachers, job placement 
specialists, and employers. 

Naturally, the specific information needs of these groups 
are dependent upon the the activities of the individuals using 
the information. Certain information components become either 
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more or less important depending on the needs and goals of the 
user. This review of the literature attempted to identify the 
information components and design considerations through an 
examination of: 

-the specific needs of New York State as defined in the 
newly developed occupational education curricula and 
guidance plans; 

-career development theory and guidance practices; 
-standards of professional associations; and 
-related research. 

It should be noted that this review did not have as its 
purpose an extensive search of the literature of career 
development theory and guidance practices, but rather was 
intended specifically to identify career information components 
and system design considerations for New York State. 

The review is divided into two major sections. The first 
section, "Career Information Components for Inclusion in a 
Computerized Career Information Delivery System," presents an 
analysis of the three occupational education curricula, an 
analysis of the guidance plan guidelines and a synthasis of 
writing on carear development, guidance and standards. The 
second section, "Design Considerations for a Computerized Career 
Information Delivery System," is composed of a similar synthesis. 
The chapter concludes with a summary list of "Information 
Components and Design Considerations for a Computerized Career 
Information Delivery System." 
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Career Information Components for Inclusion 
in a Computerized Career Information Delivery System 

Three major approaches were used to identify the career 
information components which would meet the needs of New York 
State. The first was a close analysis of the three occupational 
education curriculum guides: Home and Career Skills , Introduction 
To Technology , and Introduction To Occupations . The second was 
a similar analysis of the guidelines for guidance plans mandated 
by the new Part 100 of the Commissioner's Regulations, or the 
Regents Action Plan. The third was a reading of relevant career 
development guidance theory and research including both general 
works as well as those which dealt specifically with standards 
for and research on career information delivery systems. 
NEW YORK OCCUPATIONAL EDUCATION CURRICULA 

The three occupational education curricular programs 
examined here were pilot tested in New York State during the 
1985-86 school year. Although the structure of the programs may 
change as a result of the pilot tests, career information will 
still be needed to meet the overall goals of the program. 
Therefore, in order to present the broadest picture possible, the 
following discussion about career information required to 
complement the programs includes an analysis both of general 
needs and specific information needed. 
Home and Career Skills 

There are three broad objectives of this curriculum wh.fch is 
required of all students in grades seven and eight. The first 
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is to "Develop skills which lead to effective decision making, 
problem solving, and management in the home, school/ community, 
and workplace" (page 4.) 

One of the areas to which the principles and process skills 
are to be applied is: 

Career Planning—an overview of work, tentative plans, and 
entrepreneurship. . .Career Planning gives students the chance 
to begin making decisions and solving problems related to 
tentative career plans (page 4.) 

The "Career Planning" segment of the course "provides an 
opportunity to anticipate the future^ identify employable traits, 
examine oneself in relation to the work environment, and to 
explore career opportunities" (page 114,) It further asserts 
that "Application of the decision making process encourages 
students to form tentative career plans" (page 114.) 

Three of the seven competencies to be achieved are: 

2. Identify personal characteristics which lead to iob 
satisfaction 

3. Formulate personal criteria for career planning 

4. Apply the prociss skills to develop tentative career 
plans (page 115.) 

Scattered throughout this "Career Planning" section are 

activities which require the student to use occupational and 

career information to meet goals* Seven examples of these are: 

(1) 3. Describe characteristics of work involving various 
work locations and conditions, work personnel, and work time 
or hours. . .Several work characterisCics/environments are 
suggested. . ." 

You would like to work: 

-for people caring and serving 
-with data, facts, and records 
-witi; ideas, theories, and thought 
-with things, machines, and tools 
-outside most of the time 
-inside most of the tiaje 

ErJc 33 



CIDS Evaluation, 21 



-with other people 

-as "boss" with responsibility for others 
-following directions given by others 
-where you'd get dirty 
-where there's lots of noise 
-where it is quiet and calm 

-whera you have many decisions to make doing physical labor 
-where the work would be dangerous 
-where you would help others 
-where you earn lots of money 

-where you earn lots of money doing something you don't like 

-designing and creating things 

-doing the same thing over and over 

-at home on a flexible schedule 

-where ;j'ou need to take work home 

-on a flexible schedule 

-on "off peak" hours (page 118.) 

(2) 1. Given related checklists and evaluation aids, the 
student will apply the processes of decisionmaking, problem 
solving and management to personal career planning, with a 
degree of understanding acceptable to the instructor. 

In order to do this, the student must be able to: 

a. Identify areas of personal interests, abilities, and 
aptitudes 

b. Identify personal skills (mental, physical, emotional, 
social) r J , , 

c. Identify personal attitudes and values toward work, 
training/education, and life styles 

d. List home, school, and work experiences which contribute 
to career selection 

®' Formulate criteria for personal career planning (page 

(3) 2. After preparing criteria for personal career 
planning, the student will use the decisionmaking process to 
formulate tentative career plans... 

In order to do this, the student must be able to: 

a. Identify and use available career resources to obtain 
infoimation about careers and employment trends 

b. Identify career clusters and opportunities for the 
development of transferable skills 

c. Match career characteristics and personal characteristics 
to serve as a guide In career decisionmaking 

d. Formulate a personal tentative career plan which includes 
short-range and long-range steps needed to carry out the 
career plan (page 122.) 

(4) 1. Students identify characteristics about themselves 
which will allow them to select appropriate careers: 
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-interests, activities providing enjoyment 

-abilities, talents, aptitudes, skills 

-values, job satisfactions, experiences 

-freedom to make decisions and take responsibility 

-companionship, working with others 

-self-esteem, creativity 

-self-direction 

-self-approval, pride 

-etc. (page 123.) 

(5) 9. Have students match their personal traits, 
interests, values, and aptitudes with job tasks, 
requirements, preparation, and benefits (page 124.) 

(6) 17. Help students group job titles into workable size 
units for exploration. Emphasize the importance of 
preparing for a career cluster instead of a specific iob. . 
(page 126.) 

<7) 18. Have students practice writing a resume (page 



It should be noted here that most computerized career 
information delivery systems have a structured search routine 
designed to allow the user to search for occupations on personal 
work preferences and other variables. This is dealt with in the 
section on system design considerations. 

All the activities listed above could be enhanced by & 
computerized career information delivery system which also has 
detailed (not necessarily extensive) user materials which guide 
the students through the process of determining their own 
personal traits which might match careers and of determining 
which careers match those traits. 

Specific career information components which are needed to 
complement "Home and Career Skills" activities include: 
OCCUPATIONAL INFORMATION 

Occupational title 
Occupational description 
Occupational group or cluster 
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Job duties 
Intc^rests 
Aptitudes 
Temperaments 

Entry level skills required 

Physical demands 

Physical activities 

Environmental/work conditions 

Hours of work/ travel 

Industries in which occupations occur 

Career ladder/advancement opportunities 

Occupational outlook 

Earnings and benefits 

Education/training required 

Skills transferability 

Related occupations 

Sources of additional information 

Job seeking skills: resume writing 

Values clarification 

Interest assessment tool 

EDUCATIONAL INFORMATION 

Name and address of education/training institution 
Type of institution 
Academic/training programs of.fered 
Occupations for which programs train 
Types of degrees/certificates and length 

Introduction to Technology 

Ihe first half of this program is designed to introduce 
students in grade seven to the nature of technology and its use 
in satisfying people's needs and wants. The second half, for 
students in grade eight, focuses on: technological concepts; 
technical processes: methods people can use to control 
technological concepts and processes: technological career 
opportunities; and the ipiplication of technology to individuals 
and society. 

One of the activities listed as a "constant" occurring 
throughout the teaching of ^'Introduction to Technology" for grade 
seven is: 
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8. Career Related Information 

Information aoout career opportunities should be infused 
wherever appropriate. It should be stressed that technology 
has affected all jobs, not just those in "high-tech" areas. 
The student should be made aware at all times of 
employability skills such as dependability, honesty, 
punctuality, reliability, responsibility, ability to work 
with others, pride in work, self-awareness » self-reliance 
and self-worth (page 6.) 

The use of career information in instructional strategies is 
stressed throughout this syllabus. Specific uses include career 
awareness (defined as "a variety of experiences in which students 
learn about various career clusters, work requirements and job- 
readiness skills") and career exploration (defined as 
"investigating the necessary preparation, opportunities for jobs 
and potential for reward and fulfillment in various careers one 
might attempt to enter.") Decision making, "the process of 
identifying, analyzing and choosing among alternatives, then 
acting on and evaluating the results of the choice,") is also an 
activity for the course (Glossary, pages 2 and 3.) 

The use of occupational information for career exploration 
and decision making which is evident in the "Grade 7 Syllabus" is 
also stressed in the "Grade 8 Syllabus" (pages 3 and 41.) 

The Introduction To Technology ; "Grade 8 Syllabus" 
includes this student performance objective with supporting 
competencies: 

Given an example of a new industry in each of the three areas 
of technology, students will describe how new technologies 
have created new jobs, ... then develop a training program 
to impart the broad-based skills necessary for employment in 
one of the career clusters. 

In order to do this, the student must be able to: 

a. Identify specific new jobs in each of the three areas of 
technology and their requirements. 

b. Identify major career clusters in each of the three areas 
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of technology. 

c. List the transferable skills common to a career cluster 
(page 43. ) 

Two instructional strategies in this Syllabus are: "List 
careers and necessary training for people involved in service 
occupations" (page 30) and "Write a short career outline on 
medical careers which use new technologies to diagnose and treat 
medical problems" (page 31.) 

A computerized system which allows students to obtain 
specific information about groups of occupations could be used 
for the above activities. Career information which is needed to 
^laip meet the goals includes: 

Occupational titles 

Occupational descriptions 

Occupational groups 

Skills transferability 

Related occupations 

New and emerging occupations 

Sources of additional information 

Training programs which train for occupations 

Introduction To Occupations 

Ti is is a course required in ninth and tenth grades for all 
students planning an area of concentration in an occupational 
education area. Introduction To Occupations is described as: 
"part of a continuum of Occupational Education curriculum... 
designed to include instruction in all areas of Occupational 
Education" ("Curriculum Overview", page 1.) 

The moving force behind the establishment of this course is: 

Tie New Part 100 of the Commissioner's Regulations, approved 
November 19, 1984, makes the following statement regarding 
occupational education curriculum: 

(h) Availability of occupational education. All public 
school districts shall offer students the opportunity to 
begin an approved occupational education sequence in ninth 
grade. For students graduating in 1989 and thereafter, each 
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approved occupational education sequence shall Include a 
minimum of one unit of credit In Introduction to 
occupations, such Instruction to be offered at any point in 
the sequence (page 1.) 

Tie two purposes of this program are: "to develop 
transferable skills which can be used in later work or ome 
responsibilities" and "to explore occupational interests and 
abilities prior to taking more specialized occupational courses" 
(page 1. ) 

Two modules, "Working Citizen" and "Personal Resource 
Management", are required of all students taking the course. In 
addition, two optional modules, to be chosen from the other 
twenty-two, are required. The general needs for occupational 
data defined in the "Curriculum Overview" will be discussed as 
well as the needs identified for the two required modules. 

Of the seven proceSiJ skills in the "Curriculum Overview", 
two directly relate to occupational information and career 
information delivery systems: 

Career Options 

Career selection is an ongoing process addressing a 
comprehensive plan of flexible alternatives to deal with 
educational needs, occupational needs, and life career 
needs. Career options Include: identlf ica;;lon of the 
interrelationship of goals; values clarification; evaluation 
of interests and abilities; and career investigations (page 



Decision Making/Problem Solving 

This includes the application of decision making/problem 
solving techniques and processes to work and personal 
experiences. Students will determine short-term and long- 
term goals, address immediate needs, and identify strategies 
for successful resolution (page 6.) 

The goal of "Working Citizen" is "to Introduce students to 
the realities of the working world." 

Every working citizen si ould be able to manage information 
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and utilize resources efficiently, while realistically 
assessing his/i er own personal skills and limitations'. 
Furthermore, this knowledge and skill s» ould be applied 
appropriately in career and occupational selection, 
acquisition, and maintenance (page 2.) 

Tk e "Working Citizen" module contains five major topics, 
each of whid' "focuses on a specific area of information and 
personal development as it applies to the world of work (page 2.) 
The five topics in the module are "designed to provide students 
with information and experience which will help them make future 
d oices concerning work" (page 2.) 

Ten of the nineteen skill areas the student is expected to 
develop during this module relate to the use of career 
information. They are: 

5. Assess personal skills and talents, relating them to 
individual d'oice of employment 

6. Develop personal goals for education, training, and 
future employment 

7. List and explain personal characteristics and qualities 
that are desirable for successful employment 

8. Describe and utilize sources of employment information 

9. Determine the relationship of education and training to 
acquisition and maintenance of employment 

10. Identify at least two positions within two specific 
occupational fields of interest to the student, and describe 
the skills required for each position 

11. Identify occupational positions that require advanced 
training 

14. Analyze the working conditions of a chosen employment 
area, and explain the influence of working conditions on 
physical and mental health, as well as safety 

16. Match personal job competencies with specific 
employment opportunities 

17. Analyze the factors that influence individual job 
selection (pages 2-3.) 

In the draft outline for this Inodule, five of the major 
units require occupational and career information (pages 4-6.) 
In addition, its use is required to meet 'performance objectives/ 
supporting competencies' listed throughout the draft syllabus. 
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Alti'ough the "Personal Resource Management" module does not 
require the direct use of occupational and career information or 
career information delivery systems during instructional 
activities to meet the goals of the program, students are 
required to "identify personal goals" and to "relate resources to 
needs, wants, goals, and personal and career fulfillment" (page 
2). Certainly, the availability of a computerized career 
information delivery system for student use, especially in 
obtaining wage and salary information, could be a benefit to 
instruction. 

The other modules from which the students may choose deal 
primarily with career fields. Occupatiimal and career 
information about these fields and how to prepare for a job in 
them could be put to good use by both the teacher and the 
student. In addition, a computerized career information delivery 
system could be used to assist students in selecting the optional 
modules which might meet their needs and iriterests. 

The computerized presentation of career information, with 
routines which allow students to go through a structured search 
to find occupations which match their needs and interests; to 
obtain detailed information about occupations; to find 
occupations whid are related to others; and to make comparisons 
between occupations would be helpful to both studetits and 
teachers. 

Career information components which are directly related to 
the activities in the Introductions To Occupations curriculum 
include: 
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OCCUPATIONAL INFORMATION 

0 oupatlonal titles 

Occupational descriptions 

Occupational groups 

Job duties 

Interests 

Aptitudes 

Temperaments 

Earnings and benefits 

Environmental and work site conditions 

Physical demands 

Ei'yslcal activities 

Entry level skills required 

Education/training required and length of program 

Related civilian occupations 

Related military occupations and training 

Sources of additional Information 

Job seeking skills 

Values clarification 

Interest assessment tool 

EDUCATIONAL INFORMATION (Post secondary) 
Name and address of Institution 

Types of Institutions which offer various programs 
Academic programs offered 
Occupations for which programs train 

Types of certificates/degrees and length of time to complete 
GUIDANCE PLANS 

In November 1984, by action of the Board of Regents of the 
State of New York, a new Part 100 of the Commissioner's 
Regulations was adopted which modified the original requirements 
for district guidance plans. "The Regulation requires each sd'ool 
district to develop a district guidance plan to serve students 
kindergarten through twelfth grade" ("District Guidance Plan," 
page 1.) 

Tie Regulations state In Section 100.2 (j): 

(1) Public Sci'ools. Each sd'ool district shall have a 
guidance program for all students... 

^J:^^^^? grades 7-12, the guidance program shall Include 
the following activities or services: 

(a) an annual review of each student's educational 
progress and career plans, which such reviews to be 
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conducted with each student Individually or with small 
groups by personnel certified or licensed as sd'ool 
counselors ; 

(b) Instruction at each grade level to help students 
learn about various careers and about career planning 
skills conducted by personnel certified or licensed as 
sd ool counselors, or by classroom teachers in 
cooperation with school counselors; 

(c) other advisory and individual or group counseling 
assistance to enable students to benefit from the 
curriculum, to help students develop and implement 
postsecondary education and career plans,... 

(ill) Each sd'ool district shall develop a district plan 
which sets forth the manner in which the district shall 
comply with the requirements of this subdivision. The 
City School District of the City of New York shall 
submit a separate plan for each community sd ool 
district, for the High School Division and for the 
Special Education Division. Such plan shall be filed 
in the district offices and shall be available for 
review by any individual. The plan shall present 
program objectives, which describe expectations of what 
students will learn from the program; activities to 
accomplish the objectives; specification of staff 
members and other resources assigned to accomplish the 
objectives; and provisions for the annual assessment of 
program results. The plan shall be reviewed annuall" 
by the school districts, and revisions shall be made' as 
necessary. 

(2) Nonpublic schools. Each nonpublic secondary school 
shall provide a guidance and counseling program for students 
in grades 7-12. 

The State Education Department issued guidelines to assist 

district personnel to develop "guidance plans which meet the 

needs of their students consistent with Commissioner's Regulation 

100.2 (j)." ("District Guidance Plan, page 1.) In the Glossary, 

"Careers and ... career planning skills" are defined as 

the knowledge, understandings, and abilities related to the 
major guidance themes of self awareness/understanding, 
decision-making, planning, information gathering, values 
clarification and employability (page 4.) 

The document further states that: 

"Guidance program for all students" includes the planned 
program activities and services of personnel to enable 
students to benefit from the educational program, to develop 
the knowledge, skills and abilities necessary to implement 
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career plans, to assist in their personal and social 
development, and to develop problem solving skills and 
techniques (page 5); [and that] "Instruction at each grade 
leve^ to help students learn about various careers and about 
career planning skills conducted by personnel certified or 
licensed as school counselors, or by classroom teachers in 
cooperation with sd'ool counselors" requires that counselors 
or classroom teachers must provide planned learning 
experiences to develop the career planning skills of 
students. K'is permits career education programs and 
activities to be delivered by other than counselors. It 
requires cooperation with counselors to assure a coordinated 
approach so that counselors, in their individual work with 
students, will reinforce the knowledge and skills gained in 
instruction in the personal career planning by the student. 
These experiences may be provided to groups of classroom 
size or smaller (pages 5-6.) 

In the section of the guide which offers instructions for 
the plan, two of the program objectives for grades 7-12 relate 
directly to this review: (1) Advisory for Career Plans and (2) 
Counseling for Career Plans. The format for the plan sets forth 
five sections to be completed for each of the objectives. 

A. Program Objective; 

B. Target Population; 

C. Expected Outcomes; 

D. Annual Assessment of Program Results; and 

E. Program Description (which includes): 

1. Activities; 

2. Target Group and Sub Group; 

3. Staff Assigned and Time; 

4. Other Resources Assigned; and 

5. Dates for Activities (pages 7-8.) 

The guide offers suggestions for Section E, Activities to 
take place during grades 7-12 which might accompany an objective, 
"Instruction in careers and career planning." These activities 
include: 

Self-evaluation (grade 7) 

Use of career reference materials in library and Career 

Resource Center (grade 7) 
Identification of job clusters related to student's own 

interests and abilities (grade 8) 
Develop list of appropriate courses to be studied during 

the next sd'ool year — keeping in mind tentative plans 
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for college major and/or career goals (grade 9) 
Investigating education and training requirements for 
occupations identified in Self-Directed Seard' Survey 
form (grade 9) 

Orientation session by counselor re: Guidance Awareness 
(grade 9) 

Values Clarification exercise (grade 10) 
Decision-making exercise (grade 10) 
Self-awareness activities (grade 10) 
Career-awareness activities (grade 10) 

Group guidance sessions at which students are instructed re: 
factors important in choosing a college (grade 11) 

Group guidance sessions re: preparing and updating a resume 
for college and/or potential employers (grade 12) (pages 

/ ^ J X • / 

Since guidance plans must be reviewed each school year, the 
guide offers suggestions for "student outcomes" whic" could serve 
as the basis for the review. Some of the suggested outcomes 
which are related to career planning and decision making are: 

7th Graders will: 

o learn more about themselves by identifying their 
interests, abilities, aptitudes, values, and personal 
characteristics, and 

o be aware of their interests and of occupations related 
to their interests as derived from completion of a 
vocational interest inventory. 

8th Graders will: 

o begin preparation of their personal educational plan; 
o understand that different jobs require different 

types of education and/or training; 
o understand the basic relationships between school 

subjects and eventual occupational requirements; and 
o identify related occupations for ead' subject they 

are studying. 

9th Graders will: 

o understand that gathering information is an important 

step in educational and career planning, and 
o identify and list various types of information 
required. 

a 0th Graders will: 

o recognize a number of ways in which occupations may 
be grouped. 

11th Graders will: 

o set a long-range educational goal, and 
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o develop a written action plan for reaching the goal. 

12th Graders will: 

o demonstrate effective job Interview skills In a role- 
play situation (pages 43-44.) 

The "Review Guide" which will be used by State Education 

Department reviewers in the formal review to determine compliance 

of district plans includes the following criteria for grades 7- 

12. The district plan will be rated on each of these areas as 

"Strong," "Average," "Weak," "Unsatisfactory," or "Not 

Applicable." 

9. An ANNUAL REVIEW of each student's educational progress 
and career plans is conducted with ead student individually 
or with small groups of students by personnel certified or 
licensed as school counselors. 

10. INSTRUCTION AT EACH GRADE LEVEL to help students learn 
about various careers and about career planning skills is 
conducted by personnel certified or licensed as school 
counselors, or be classroom teachers in cooperation with 
school counselors. 

12. Other ADVISORY ASSISTANCE to help students develop and 
Implement postsecondary education and CAREER PLANS is 
provided by teachers or counselors, or be certified teaching 
assistants under the supervision of counselors or teachers. 

17. Other individual or group COUNSELING to help students 
develop and implement postsecondary education and CAREER 
PLANS is provided by certified or licensed school 
counselors, or by certified or licensed school 
psychologists, or by certified or licensed school social 
workers in cooperation witi school counselors (pages 46-47.) 

It is obvious from reading Comr ^'sloner's Regulation 100.2 

(j) that a wide variety of career planning information is needed 

to satisfy the intent and objectives as set forth. Certainly the 

availability of a computerized career Information delivery system 

for use as a resource tool would be extremely useful. T/'e State 

Education Department's Guide to school districts for developing 

their guidance plans make it clear that the schools must have 
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information about careers available to students, counselors, and 
teachers. In addition, they must have adequate materials and 
systems in place whid' can support a broad range of actual career 
planning and decision making activities. 

Althou^ the need for and use of all types of career 
planning information are evidently needed to support the guidance 
activities required of ead' school district, specific information 
components include: 

OCCUPATIONAL INFORMATION 

Occupational title 

Occupational description 

Occupational groups or cluster 

Inteiests 

Aptitudes 

Temperaments 

Entry level skills required 
Job duties 

Education and training required 
Related secondary school subjects 
Related occupations 
Job seeking skills 

Links to published assessment instruments 

EDUCATIONAL INFORMATION 

High school areas of concentration 

Name and address of colleges or training facilities 

Admission requirements 

Education and training programs offered 

Occupations for which education and training programs train 
Financial Aid information 

THEORY, STANDARDS, AND RESEARCH 

In 1951 the National Vocational Guidance Association (which 
is now the National Career Development Association) amended its 
definition of vocational guidance. TVe earlier definition stated 
that vocational guidance is "the process of assisting the 
individual to choose an occupation, prepare for it, and progress 
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in it". This definition was changed to read "the process of 
helping a person to develop and accept an integrated and adequate 
picture of himself and of his role in the world of work, to test 
this concept against reality, and to convert it into a reality, 
witi' satisfaction to himself and society" (Herr and Cramer, 
1979.) 

Herr wrote that this new definition blended the personal and 
vocational dimensions of guidance and "focused on self- 
understanding and self-acceptance as the evaluative bases to 
whio." can be related the occupational and educational 
alternatives available to the individual" (Herr, 1980.) Hoyt 
emphasized the career education needs of the individual as 

the total effort of public education and the community to 
help all individuals become familiar with the values of a 
work oriented society, to integrate these values into their 
personal value systems, and to implement these values into 
their lives in such a way that work becomes possible, 
meaningful, and satisfying to each individual (Hoyt, 1974.) 

Ti ere are many different vocational development theories, 
and much has been written about them. For the purpose of this 
discussion, we will consider Lawrence and McAdams' (1978) 
definition which places counseling development theories into 
three categories: indJvidual-centered theories, sociological and 
economic theories, and accident theories. (Accident theories 
will not be examined here. ) 

Lawrence and McAdams categorized individual-centered 
theories in two areas: developmental ar.i non-developmental. 
Vocational developmental theories, set forth by Ginzburg, 
Tiedeman, and Super, emphasize the individual's development of a 
self-concept and the decision making process which results in the 
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individual's selection of an occupation. Holland's theory, on 
the other hand, may be considered non-developmental In that It 
focuses on the individual's identification of key traits and 
finding jobs that fit these traits. 

Ti'e sociological and economic theories of vocational choice 
described by Datta emphasize the sociological context and 
environmental factors within which a person makes a decision 
rather that the decision making process itself (Datta, 1975.) 

Tt'e person's need for career information in career decision 
making does not fit neatly into these theories. Even though the 
theories of vocational development are important in that they 
enable one to look at how people make decisions, it must be noted 
that there appear to be two gaps in the basic research: the absence of 
specification of occupational information whid' a person needs to 
make a choice, and the absence of a description of the 
process by which people combine information about careers with 
information about themselves to come to a choice. 

Bhaerman wrote about the use of occupational information and 
computerized systems in career counseling: 

One of the most important— but also most neglected— methods 
ot career counseling is the presentation and use of 
occupational information in deci^on making. Not only are 
career counselors often overwhelmed by the sheer volume of 
information that is available, but often they have 
difficulty in knowing how to impart it to clients... 
Computer-assisted occupational information systems have 
Introduced a new and more interesting mode of presentation 
(Baehrman, et.al., 1982.) 

Were are many models for structuring career guidance 
activities. Gysbers (1983) i:'entified seven phases in career 
decision making: (1) exploration; (2) crystallizaton; (3) choice; 

ERIC 49 



CIDS Evaluation, 37 



(4) clarification; (5) induction; (6) reformation; and (7) 

integration (page 11). In discussing the career decision 

making steps in the formulation of an individual career 

development plan for students, he stated that: 

...there is actually much interaction among the phases. 
T^'ere probably is a good deal of starting, stopping, going 
back, and repeating behavior- It is important to remember 
that the behaviors involved are learned; hence you (the 
counselor) may need to provide experiences for your 
clientele to halp them adopt, adapt, or create a career 
decision-making approach that makes sense to them. 

TMs advice makes a good argument for the inclusion of a 

computerized career information delivery system as a counseling 

tool to assist both young people and adults in the process career 

exploration and decision making. The question here is: What are 

the occupational and educational information components to be 

Included in a system which will meet the needs of youth, 

secondary and postsecondary students, the unemployed, the 

incarcerated, entrants and re-entrants to the job market, people 

making mid-life career changes, military returnees, and others? 

Some specific needs of students have been discussed in the two 

preceding sections in relation to the new occupational education 

courses in New York State and the State guidance plan 

requirements. 

Two relevant studies were conducted in New York State. 
Heller, Wheeler, and Gross investigated the need for, and 
interest in, computer-based guidance in secondary, postsecondary, 
and correctional institutions. They surveyed all community 
school districts In the State, excluding those comprising the New 
York City School System; 57 public and private two-year colleges, 
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and eight Educational Opportunity Centers and correctional 
facilities. 

Ti'ey found that half the respondents from school districts 

were using an automated system during 1978-79 and that an 

additional 30 others, even though they were not using such a 

system, had experience with them. One-fourth of the 

postsecondary respondents were using a computer-based system for 

guidance. The computerized systems which were identified as 

being used at that time were CVIS, GIS, SIGI, Discover, and CIS. 

One of the study results which has implications for the 

current investigation is the answer to the question about whether 

guidance specialists had an interest in a statewide system. 

Half of the respondents indicated that they would be 
interested in participating on one or more committees that 
we proposed to investigate topics associated with this 
endeavor. Most, of course, were interested in working on 
content issues — that is, what are the needed typ-s of 
information; and fewest were interested in working out the 
hardware and communications configurations (Heller Wheeler 
and Gross, 1979.) 

Tie second study was conducted for the New York Education 
Information Centers Program to "provide information useful in 
developing and implementing a computerized career information 
system vhose primary service audience would be adults." One of 
the research activities, "to identify specific occupational, 
educational and related information needed by adults encountering 
a variety of career choice-making situations", has relevance for 
this study. The assumption is that these adults include tJJose 
who are unemployed, are entering or reentering the job market, 
are making voluntary mid-life career changes or are displaced 
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workers (Franklin and Mayall, 1981.) 

X/'e information components which were deemed to be necessary 
for adult career decision making were selected and grouped into 
three categories. Ti'ree panels assessed how well three systems 
(CIS, CBEOC, and GIS) could provide the needed information. The 
information components were: 

OCCUPATIONAL INFORMATION 

Structured occupational search strategies 

Job description information 

Job requirements information 

Licensure and certification information 

Education and training source information 

Occupational outlook information 

Job-seeking skill information 

Earnings and fringe benefit information 

POSTSECONDARY INSTITUTIONAL INFORMATION 

General information 
Cost and refund policy 
Financial aid information 
Information on admission requirements 
Academic program information 
Assessment of prior learning 
Information on schedule flexibility 
Information on institutional support services 

NON-COLLEGIATE INSTRUCTION INFORMATION 

Financial aid information 

Union/ apprenticeship training information 

Adult basic education program information 

Day care/evening care information 

Information on support services for adult learners 

Information on proprietary institutions 

Assessment of prior learning (Franklin and Mayall, 1981.) 

This information could also be useful to military returnees. 
People who have received Job training in the military services 
also need to have information about civilian occupations which 
are related to the military occupations they may have held. A 
recent initiative by the Department of Defense has provided 
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military career information for incorporation into computerized 
career Information systems. HVey hope to encourage young people 
to consider the military services as a career. The Military 
Career Guide (1984) points out that "most military occupations 
are comparable to one or more civilian occupations because they 
require similar duties and training." Information about skills 
transferability from military occupations to civilian occupations 
is helpful both to youth d'oosing a career and to military 
returnees. 

State Employment Security Agencies in every state manage the 

employment service and the unemployment insurance operations, 

which are designed to serve all citizens who are out of work and 

looking for work. Odell stressed that it is important for the 

employment service to possess end use good occupational labor 

market information in serving their clients, not only in the area 

of job placement but also in counseling and assisting clients to 

explore training opportunities. He pointed out that 

there are hundreds of thousands of people . . . who need 
positive assistance including sound occupational and labor 
market information in order to survive and prosper in a 
changing world and a changing labor market.... High 
quality counselors with access to a full range of high 
quality occupational and labor market information are needed 
in the employment service and all employment and training 
agencies.... It is clear than an employment service which 
is competent to Intervene on a large scale to provide 
vocational guidance and counseling services to youti' and 
adults must have substantial occupational Information 
capability of its own or must have ready and speedy access 
to some other agency's system if it Is to Intervene 
successfully in its role (Odell, 1981.) 

One of the needs Odell stressed is for a computerized job 
bank system and a person-job matching capability. He also stated 
that the "employment service should have a reasonable 
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comprehensive and current fix on the demand for labor In each 
local office and labor market area." In addition to the job bank 
and person- job matching Information, he listed the types of 
Information the employment service needs to support counseling, 
placement, and related activities. They Included: 

1. Occupational Information concerning short- and long-term 
labor demand on a labor market area level. 

2. Occupational Information concerning short- and long-term 
labor supply on a labor market area level. 

3. An annual labor market area analysis of supply and demand 
by Industry and occupation (Odell, 1981.) 

Both adults and youth with handicapping conditions have 

special career Information needs, and some of them need alternate 

delivery methods for computerized career Information. Morlarlty 

advanced the following Issues In regard to the needs for 

vocational rehabilitation. 

1. Occupational Information dealing with the work setting, 
aptitude and related Issues needs to become bidirectional, 
concerned with how jobs can be accommodated to Individuals 
and not just the other way around. 

2. In the "best situation," accommodation Information would 
be an Integral part of the occupational information system 
itself. Here a computerized data bank cross-referenced to 
job titles can be envisioned with continual updates as new 
information and technologies become available. 

3. In addition, "exemplary practices" could be stored 
depicting how jobs may be accommodated to handicapped 
persons through restructuring, job sharing, use of flextime, 
and other management techniques. 

4. The benefits from the above would accrue not just to 
handicapped person, but to generalized improvement in the 
understanding of the relationship between worker and job and 
the impact of accommodation on both (Morlarlty, 1981.) 

In the area of career counseling for vocational 

rehabilitation clients, the following information components were 

identified as being needed at both state and local levels: 

Current occupational demand 
Projected occupational demand 
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Expansion or reduction in occupational demand (into the future) 
Replacement demand 

Employment demand (localized by industry) 
Occupational supply 
Occupational characteristics 

According to Moriarity, occupational characteristics are 

"probably the most valuable component for VR" and Include: 

Earnings and benefits 
Job duties 

Work settings and conditions 

Abilities and aptitudes 

Licensing/ registration requirements 

Education/training skill and precision patterns 

Education/ training requirements 

Demographics of workers 

Career ladders 

Major employers 

Hiring channels 

Occupational titles and descriptions (Moriarity, 1981.) 

Seigel and Buchanan (1981) stressed the need for very 
detailed and accurate data about the physical requirements of 
jobs for vocational rehabilitation career planning. They also 
stated that "This data must be presented in such a way that the 
requirements of the job can be related to information about the 
client. 

For the midcareer physically disabled, McMahon recommended 
the use of occupational information in relation to three 
occupational choice factors: self-assessment, understanding the 
realities of work, and decision making (McMai'on, 1981.) 

•a'roughout the literature dealing with rehabilitation 
counseling for people with learning or reading disabilities, the 
point is made that the reading level must be such that the user 
can make maximum use of the materials or systems witi'out being 
"shut out" by the information's high reading level. 

During the counseling process, Moriarity saw "no more 
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useful tool to client/counselor than a good occupational 
information system." On the issue of delivery of information to 
the people with visual and hearing impairments, he added "the 
computerization of occupational information paves the way to use 
this technology for alternative sensory display: print tor the 
hearing impaired and audio (digital and synthesized voice) for the 
visually impaired" (Moriarity, 1981.) 

Keim reported using videotape presentations in a career 
awareness program. The presentation of computerized vocational 
guidance information to people with handicapping conditions 
warrants special consideration by any group seeking to establish 
a system to serve them (Keim, Rak, and Fell, 1981.) 

A study by the Florida State Occupational Information 
Coordinating Committee reiterated the need for the above 
information components and stressed the use of computerized 
systems for i(se in career counraling for physically handicapped 
individuals (Florida SOICC, 1981.) 

Reardon discussed the use of computers to assist handicapped 
people to move beyond the stereotype that there are only a 
limited number jobs that they can perform. 

Many of these systems help users assess their interests and 
competencies, learn decision-D.aking skills, and explore 
occupational information. . .Thty can empower a client to 
explore remote aspects of vocational behavior with the 
expenditure of little time or effort. . .Properly designed, 
these systems can remove some of the stereotypes that are 
barriers to handicapped persons' free exploration of 
occupational alternatives (Reardon, 1981.) 

In recent years, there has been an emerging interest in 
women seeking to enter or reenter the job market. A November, 
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1985 issue of Ms magazine article identified categories of 
information useful for this group. 

If working women are going to find jobs for the future, they 
are going to need tools to help them. . .choose careers that 
will meet their needs and interests. (H"ere are) three 
kinds of information needed to evaluate a particular 
occupation. First is the Costs of training and skill 
development and Benefits in salary and advancement 
opportunities of the occupation. Ti'e second focuses on Job 
Availability.... The third (is) on-the-job Working Conditions 
(Mutari, 1985.) 

Tl'e article also listed specific data categories within these 
broad headings: 

Skills required 

Training requirements 

Where the training is available 

Training costs 

Salary ranges 

Career Mobility 

Number employed 

Projected growth 

Geographic concentration 

Traditional/pathbreaking (information about occupational 

segregation by race and sex) 
Occupational interest groups: professional organizations 
Work schedules 
Degree of supervision 
Contact with public: with coworkers 
Control or pace of work 
Physical mobility 

Health and safety (environmental conditions) 
In her book, Re-Entering . Herman suggested that women need 
more than good occupational information. Ti'ey need professional 
career counseling to help them sort out their needs and values; a 
workable career plan; and job seeking skills, such as resume' 
writing and interview techniques (Herman, 1980.) 

In an effort to make computerized career information 
available to the general public, b • especially to all adults, 
several operators of CIDS programs nave placed their system in 
public libraries. "The San Hernadino, California Public Library 
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reports a 'tremendous success' to the introduction of its Eureka 
computerized data base" ( Library Journal . November, 15, 1982.) 
■M e Georgia Career Information System is being used extensively 
and successfully by people in the Atlanta Public Library ( Library 
Journal, March 1, 1982.) 

In a recent article, "Matching Yourself With the World of 
Work," Fountain stated that it is helpful for career choosers to 
know as much as possible about what an occupation demands. He 
presented a guide for exploring occupations and comparing job 
characteristics with the reader's interests and skills. The 
guide listed 16 occupational characteristics related to more than 
250 occupations. K'e characteristics he listed as important for 
exploration and comparison are: 
JOB REQUIREMENTS 

Leadership/persuasion 
Helping/instructs others 
Problem solving/creativity 
Initiative 

Work as part of a team 
Frequent public contacts 
Manual dexterity 
Physical stamina 

WORK ENVIRONMENT 

Hazardous 

Outdoors 

Confined 

OCCUPATIONAL CHARACTERISTICS 

Jobs concentrated geographically 

Part-time 

Earnings 

Employment growth 

Entry requirements (Fountain, 1982.) 

Sun and Harrington wrote about the occupatiorjal Information 
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needs for CETA program operations. Even tiioug*^ CETA has been 

replaced by JTPA (the Job Training Partnership Act), the 

information needs for counseling and placement of employment and 

training program clients have not been replaced. They addressed 

the total information needs for all aspects of the CETA program, 

but some of their comments have direct bearing on "counseling and 

guidance to job-seekers." They state that: 

CETA ... program managers must have access to timely and 
reliable information on labor force, aggregate employment 
and industrial and occupational employment developments in 
their planning and local labor areas.... Information on 
aggregate employment levels and trends in local labor 
areas as well as information on growth industries and the 
occupational staffing patterns of local growth industries to 
allow ... for the provision of institutional and OJT 
training; Information on the hiring requirements and hiring 
policies of local firms to design appropriate curriculum for 
training programs and to provide relevant occupational 
counseling and guidance to job-seekers; and information on 
the hiring activities of local firms to facilitate the job 
placement of terminees from ... employment and training 
programs (Sum and Harrington, 1981.) 

Baker discussed some of the information needs of the 
Job Training Coordinating Council (JTCC) in Kentucky. The 
information he said the JTCC needs there is obviously needed in 
most states. Among the questions he raised relating to program 
plGiining and counseling operations are: 

What occupations are in demand? 

What training should be provided for the economically 
disadvantaged or dislocated worker population? 

What are the characteristics of the labor force as a wrf)ole, 
particularly the economically disadvantaged? 

How many persons are trained? 

How many of these receive jobs? (Baker, 1984.) 

Even though the needs for specific information components of 
JTPA clients are not significantly different from other people in 
their age groups, it must be remembered that they often have 



Er|c 59 



CIDS Evaluation, 47 

educational and social disadvantages that which be taken into 
account when considering their career decision making process 
needs . 

New York State has a training program for dislocated workers 

which provides these workers up to $1,500 in tuition assistance 

for approved training programs. According to DeFabio the State 

legislation whid' established the program provides that the 

Commissioner of Labor produce a list of occupations likely 

to be available in the state for the next three years 

Tl e dislocated workers are provided with a list of the 27 
occupations in which the average annual openings are 
expected to exceed 1,000 for the next three years, according 
to OES projections (DeFabio, 1984.) 

Ti'e need of these workers for other types of Information 
concerning the occupations was not addressed in the legislation, 
but it is certain that they would benefit from having additional 
information about their potential new jobs. 

Admittedly, certain population groups certainly need 
specialized types of occupational and education and training 
information to aid in career decision making, there are many 
aspects of the process that J'old true across all groups. Maze 
and Cummings pointed out that "Since choice points occur several 
times in each person's life and since they are spaced so far 
apart in time, oae of tha most effective ways to assist clients 
is to teach them the career planning process." 

They list five steps in the career planning process: 
STEPS POSSIBLE ACTIVITIES 

I. Personal Review Analysis of past experience 

Career pati' assessment 
Identification of needs 

II. Assessment Goals 

Values 
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V. 



IV. Planning 



III. Decision Making 



Implementation 



Interests 

Personality 

Skills 

Aptitudes & Abilities 
Motivators 
Identify options 
Gather information 
Assess desirability of options 
Assess risks 
Select goal 
Assess skills needed 
Identify training opportunities 
Identify resources for job openings 
Outline plan 
Education or training 
Volunteer, temporary, or part time 
jobs 

Professional organizations 
Personal development workshops 
Networking 

Job search (Maze and Cummings, 1982.) 



The remaining portion of this section will focus on 
recommendations for information components to be included in 
computerized career information delivery systems which can aid in 
the overall career planning process. 

A Norti' Carolina Study identified labor market information 
needs for seven education and training related activities. One 
of those was "vocational or career counseling". The state 
agencies surveyed were: Public Instruction, Community Colleges, 
CETA, Vocational Rehabilitation, and the Employment Service. The 
data elements which were ranked either "of critical importance" 
or "of above average importance" for vocational or career 
counseling by three or more agencies include: 
CHARACTERISTICS OF TYPICAL JOB 

entry wage rates 

average wage rates 

job duties and responsibilities 

working conditions andJ'ours 

linkage with other jobs 

hiring channels customarily used 
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licenses, accreditations, certificates required 
apprenticeship opportunities 
training opportunities 
fringe benefits 

EMPLOYMENT PATTERNS 

seasonal 
temporary 
permanent 
full-time 
part time 

JOB OPENINGS 

by industry 
by occupation 

by occupational cluster, etc. 
EMPLOYMENT REQUIREMENTS 
CHARACTERISTICS OF WORKERS 

traits and qualities generally associated with employer 
acceptance 

OCCUPATIONAL MOBILITY 

EDUCATION AND TRAINING PROGRAM CHARACTERISTICS 

program title 
program description 
eligibility requirements 
identification of program providers 
budget 

support services offered 

financial aid (Probst, et.al., 1979.) 

In a career information system feasibility study for the 
State of Illinois, Rath identified virtually the same information 
components as being needed for a state-wide system there (Rath, 
Jacobson, and Grabowski, 1980.) 

Since 1981, NOICC, in cooperation with the U.S. Department 
of Labor, Employment and Training Administration, and the 
Department of Defense have funded states to conduct a series of 
local training worksi'ops on "Improved Career Decision Making 
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Through the Use of Labor Market Information" (ICDM). These 
workshops are conducted to assist counselors in learning where to 
find and how to use labor market information to help their 
clients make career decisions that reflect the realities of the 
labor market. 

Tl'e categories of labor market information which are 

discussed during these ICDM training sessions include: 

Occupational activities 
Occupational characteristics 
Preparation for work 
Job advancement 
Related occupations 
Industry information 
Employment outlook 
Earnings 

Places of employment. 

The workshops are designed for counselors working in high 
sd'ools, the Job Service, JTPA erly CETA) programs, 

vocational rehabilitation centers, . nd community and technical 
colleges, as well as other agencies which offer counseling 
services in a state. 

In 1977, Ti'e U.S. Department of Labor issued a framework of 

standards for the development of "systems to provide career 

information to person who are in the process of career 

exploration and decision making." In relation to information 

to be included, the publication states: 

People wl'o are in the process of career exploration and 
decisionaaking need a great variety of occupational, 
educational, and training data and information. An ideal 
system should contain descriptive materials (about)...: 

OCCUPATIONS 

DOT codes... or combinations. 

A description of duties or tasks (nature of the work) . 
A description of special tools, equipment, or instruments 
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used on the job* 
Identification of other occupations that have similar 

skill or knowledge requirements (skill transferability) 
Opportunities for promotion or career advancement. 
Information on hiring channels. 

Working conditions (indoors or outdoors, work week and 
schedules, and working conditions sud' as stress, 
physical settings, safety, etc.)... 

REQUIREMENTS OF THE OCCUPATION 

Personal requirements 
Interests 
Aptitudes 
Abilities 

PJ»ysical qualities that can be related to characteristics 
of the occupation... 
Preparation requirements 
General education. 

School uubjects and courses of study. 
Special training or education. 
Work experience. 
Other requirements: 

Licensing or certification information. 

Information on associations or unions. 

Information on examinations... 

Information on special requirements, suci' as 
citizenship, language, etc. 

ECONOMIC INFORMATION ABOUT THE OCCUPATION 

TVe number of workers in the occupations and related 

data — industry employment, geographic distribution, 

self-employment, etc. 
Descriptive outlook information 

Projections of demand 

Supply 

Relationship between supply and demand 
Factors that affect outlook... 
Information and data on earnings 
Beginning earnings 
Average earnings 
Ranges 

Information on fringe benefits... 
Information on costs to workers: union membership, tools, 
and equipment, etc (U.S. Dept. of Labor, 1977.) 

The basis for the recommendations for the inclusion in a 

computerized career information delivery system of the 

information components listed above are based on the findings of 

the seminal work by McKinley in 1974 in developing a computerized 
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career information system in Oregon. It is safe to say that mosl 
of the computerized systems in use today are organized around 
McKinlay's design. For that reason, one will find, in general, 
similar information components in most of the computerized 
career information delivery systems being used today. 

The Association of Computer-Based Systems for Career 
Information (ACSCI) has issued a publication dealing with 
standards for career information delivery systems. The document 
deals with criteria to be used for the establishment of a high- 
quality system which meets the needs of a broad base of users. 
The standards in the publication deal with the all aspects of the 
development and delivery of computerized career information. In 
this section, those standards which deal with information are 
reviewed. 

The ACSCI standards identify the following information 

components as necessary in a career information delivery system: 

Information which describes the occupation should be 
adequate to give a complete description and should include: 

The occupational title and an identifying code. 
The primary job duties of the occupation should be 
described. 

Abilities required for worker success in job. 
Skills transferability of an occupation. 
Information about related occupations. 
Working conditions of the job. 

Equipment, tools, or instruments used on the job. 
Wage information which includes beginning salary; 

average salary for experienced workers; salarv ranges; 

fringe benefits; cost to workers for union membership, 

etc, all of which are presented consistently. 
Employment outlook information and the relationship 

between supply and demand. 
Interests. 
Aptitudes. 
Physical qualities. 
Preparation requirements. 
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Licensing, certification, or examinations required 
for entry. 

Opportunities for special groups, such as minorities 

and handicapped si'ould be included. 
Common methods of entering the occupation. 
Career ladder and advancement information. 
Apprenticeship information. 

Location of educational institutions where training 

can be obtained. 
Financial aid information 
Job search information 

Educatioji and training information specific to the 

needs of special populations s."ould be presented (ACSCI, 1981) 

Ti'e National Occupational Information Coordinating Committee 

(NOICC) has issued "Career Information Delivery Guidelines" for 

'^»:e Occupational Information Coordinating Committees (SOICCs) 

*o for the development, delivery, and use of career 

-oruiation. "H'e section of the Guidelines which address the 

iniorma^f.on files states the "a system si'ould develop information 

which adequately describe.-, the occupation." The rec mmended 

contents of the information files include: 

OCCUPATIONAL INFORMATION FILE 

Descripvive information about both civilian and military 
occupations, such as: 

Definition of the occupation; 

Job duties; 

Working conditions; and 
Career ladders. 
Requirements of the occupation, such as: 

Education and experience required for entry; 
State and local licensing requirements; 
Interests ; 
Abilities; and 
Temperaments. 

Economic information about the occupation, such as: 
Earnings ; 
Employment ; 

Industrial concentration; 
Sources of labor supply; 
Employment outlook; and 
Supply and demand analysis. 

EDUCATION AND TRAINING INFORMATION FILES 
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Postsecondary educational programs; 
Educational Institutions (schools and colleges: 
Military training; 
Apprenticeship ; 
Sc/'ool subjects; and 

Financial aid Information (NOICC, 1985.) 

In addition to discussions in the literature about career 
planning Information needs, two topics have emerged which should 
be mentioned here. These deal wlti' entrepreneurshlp and new and 
emerging occupations. 

An Interest In entrepreneurshlp has always been a part of 
some peoples' career planning. This has grown rapidly In recent 
years and Is expected to continue to Increase. Cetron points 
out: 

In 1950, fewer than 100,000 new businesses took form, 
compared with 700,000 in 1983. Self -employment has hit boom 
times as well. After a 20-year decline, the number of 
people self-employed Increased by 25 percent during the 
1970s and probably will double In the next decade. 

We predict the movement Into the Information Age will 
provide a fertile environment for entrepreneurs. . .Look for 
new business formation to boom and, with ^t, look for some 
very successful, very happy and very rich young business 
owners (Cetron, 1985.) 

A career Information delivery system which Is designed to 
meet the needs of these potential "young business owners" (and 
older ones as well) should Include Information about the types of 
occupations and training programs most likely to lead to 
entrepreneurshlp and self-employment as well as personal traits 
whid' entrepreneurs si'ould possess. 

Another topic which appeared in the literature with 
Increasing frequency is "new and emerging occupations;" also 
referred to as sunrise occupations, jobs of the future, or new 
technologies. According to Cetron (1985), it is important to 
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understand trends in future job growti' and to develop personal 
forecasts. "The Information Age that is so drastically altering 
our workplace also can give the average individual information to 
help him or her shape a safe, sane and reasonable future." 

Borchard (1984) pointed out that "Career planners need to 
tune in to the future if they are to remain equipped to assist 
today's career choosers and charters." He assessed current 
computerized systems as "inadequate sources of job information" 
for people who want to train for careers which will be in demand 
into the next century. "Most counseling tools are showing their 
age when it comes to assisting people with career planning toward 
the year 2000." 

Professional labor economists and come others who are 
writing about the labor market today are saying that the needs 
and make-up of the labor force in the near fut'are will be 
significantly different in some work areas from today's. If a 
computerized career information system is to be of maximum use to 
tl'ose it is intended to serve, it must be designed to include 
information about new occupations, the changing needs of the 
labor market, and have the ability for new types of information 
to be incorporated. In his article, "Careers of the Future," 
Wolkomir (1985) pointed out that "in an a^s of rapid advance, 
careers are likely to change as frequently as the technology on 
which they are based," This would make even more important the 
contention that people need not only information but also career 
planning skills which they can use again and again. 

It is interesting to see what is being written about future 
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jobs. Wolkoiair listed ten job categories which, in tW long-term, 
will be doomed to obscurity and ten on the rise. He stated that 
those jobs on the way out include: factory workers; clerks; 
middle management, income-tax specialists, computer programmers, 
air-traffic controllers; other professionals; farm laborers; 
lumber jacks; and fast-food workers. He cited the rise in 
intelligent computer software and the use of robots as two causes 
for the decline. The ten "on the rise" in the long-term are: 
fire-protection engineers; database managers; aquaculturists ; 
laser technologists; space technologists; professional humanists; 
digitechnicians; educationists; geriatricians; and artificial- 
intelligence engineers. 

Bolles, however, reminds us that as we plan for the future, 
we si'ould not forget that "there will be constancies as well as 
change. . .constancies which will be transport^-d untouched and 
unchanged into our future". He lists sever/^l types of workers 
which represent these constancies which include those who make 
our shelter for us; ti'ose vi;o repair out vehicles and appliances; 
and those in service occupations. Even though the skills these 
workers need may be different from present skills required 
because of changing technology, the personality types of the 
people wi'o will be attracted to these jobs is likely to be the 
same. 

Bolles recommends "life/work planning" which will allow 
people to take stock of what within them is constant and what has 
changed and to apply that information to the labor market. He 
lists seven major categories of information to be considered as a 
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"constant or change" when a person engages in the activity of 
"life/work planning." These are: geography; goals; special 
knowledge; people environments; working conditions; level and 
salary; and transferable skills (Bolles, 1983.) 

The purpose of this review has been to investigate the 
information component needs of users of a computerized career 
information delivery system for the State of New York. Several 
interesting facts emerged during the review. No single b/fjok, 
article, or other source addressed all the issues relating to the 
use of occupational and educational information in career 
decision making and in connection with computerized career 
informvation delivery systems. It is interesting to note th&c the 
literature which dealt with theory, for the most part, did not 
deal with information needs. On the ocher hand, in general, 
d'ose articles which recommended a wide range of career decision 
making information tended to avoid any discussion of theory. 
Research which -«ould directly link theories of career decision 
making with information needs has been neglected and needs to be 
undertaken. However, a consistent range of occupational and 
educational information components did emerge from this review. 
A list of the components can be found in the chapter summary. 
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Design Considerations 
for a Computerized Career Information Delivery System 

There are only a few publicatiors which discuss standards 
for and issues related to the design of computerized career 
information delivery systems. One of the earliest to begin to 
address this issue is the 1974 publication by McKinlay, which 
listed the "necessary features of an information delivery system 
for career planning." He characterized "an ideal occupational 
information delivery system" as one that can: 

1) Make information accessible to persons of varying 
ability and experience. 

2) Provide a mv?ans for integrating occupational information 
with clients' interests, values, aptitudes, and abilities 

J J Use the media that are most appropriate for the 
particular finds of information. 

4) Display and/or deliver information in an attractive 
manner . 

5) Provide accurate and current information, including 
capacity for updating. 

6) Supply local as well as national data. 

7) Provide information concerning a wide variety or 
occupational groups. 

8) Include such specific information as: (a) job duties, (b) 
work environments, (c) hiring and training requirements, 
(d) terms of employment, (e) hours, (f) current labor 
?Q7A ? ®^^"^^^°"» and (g) lonfe-range outlook (McKinlay, 

In 1977, a U.S. Department of Labor, Employment and Training 
Administration bulletin discussed the delivery component of a 
career information system. Even though these standards were 
published nine years age, they still are very relevant to the 
developer and user of coicputerized career information delivery 
systems today. 

The delivery of career information to users includes two 
broad functions, accessing and dissemination. Accessing is 
the strategy or approach used by the individual to search 
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out and explore Information. Dissemination Is concerned with 
the provision or availability of the Information for a wide 
variety of users (U.S. Dept. of Labor, 1977.) 

Tl'ls publication set forth "operating standards for 

accessing and dissemination [which] al'ould ' e met by a career 

information system." These Include: 

o The delivery system si'ould be demonstrably effective with 

persons of varying ability and experience, 
o System components si'ould be user-operable so that 

Independent usage may be fostered. .. .Hl'e client will need 

little or no assistance In using them, 
o A booklet or set of Instruttlcas on how to use the 

various components si'ould be i callable to clients using 

the system. 

o Varied media should be Uced to disseminate Information. 

... The various media, .however, si'ould be components of 

an Integrated delivery system, 
o Hardware components required si'ould be standard equipment 

that Is reliable, widely available, and applicable to a 

variety of users, 
o Ti'e delivery system must be able to provide a copy of a 

summary of its basic information to the m-^-r , 
o Ti'e delivery system should be able to house all 

information related to the topics listed for information 

development, including localized information, 
o The system must have an updating capability so that new 

information can be entered Into the system and made 

available to users within at least 6 months of the time 

it becomes available, 
o The user should be able to obtain the in ^rmatlon desired 

before leaving the system, 
o Ti'e accessing strategies or approaches used to obtain 

Information a/'ould be available both in a "direct access" 

and a "structured" search mode, 
o The delivery system si'ould be available as part of the 

counseling and instructional processes, 
o The systea si'ould be as accessible as possible for use 

throughout all or most of a user agency's regularly 

scheduled day. 

o Costs for the delivery system should be at a level per 
user which vtll make it feasible for the user agencies 
or local or State governments to support them 
financially (U.S. Dept. of Labor, 1977.) 

Ti'e NOICC Guidelines stated that accessing strategies for a 

computerized career information delivery system si'ould include 

both: 
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o Direct search - This is a meti'od for users vi'o know the 
name of a specific record: e.g., school or occupation, and 
use the name to "look up" and obtain information on this 
record by using a file title index. 

o Structured search - This is a meti'od for users wi'o do not 
know which record (usually an occupational title) to 
explore. The structured search provides such users with 
a sorting mechanism by which they may identify and record 
their personal preferences and obtain a list of related 
occupational titles to explore (NOICC, 1986.) 

The Association of Computer-Based Systems for Career 

Information (ACSCI) also has issued standards for a computerized 

career information delivery system. These include: 

o The delivery system should be demonstrably effective with 
persons of varying ability and experience. 

o Tl'e delivery system should present information in an 
attractive and interesting manner so as to motivate users 
to continue use of the system and to further explore 
occupational areas. 

o The delivery system components should be user-operable so 
that independent usage may be fostered. 

o The system should provide user agencies with booklets or 
sets of instructions Gti J'ov to use the various components 
for distribution to userSr Such Instructions should 
include directions for operating the various hardware 
components and descriptions of the information files 
available in the system. The Instructions s^'ould be 
written at a reading level appropriate to a wide variety 
of intended users. 

o The statement made in one medium should be consistent and 
compatible with ti'ose made in other files or media. 

o Any delivery system hardware components required should be 
standard equipment that is reliable, widely available, and 
applicable to a variety of users. 

o The delivery syst .js must be able to provide a copy of a 
summary of its basic information to the user for future 
reference. 

o The delivery system should be able to house all information 
related to the topics listed for information development, 
inclxiding localized information. 

o The system must have updating capability so that new 
information can be entered into the system and made 
available to users within at least six morths of the time it 
becomes available. 

o The user should be able to obtain the infccaation desired 
which is available through that medium of delivery before 
leaving the system. 

o Ti'e accessing strategies or approaches used to obtain 

information si 'ould be available both in a direct access and 
a structured search mode depending on the preferences of 
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the user, 

o If the accessing strategy provides for the use of a 

client's records (e.g., test scores, class rank. Interest 
Inventories, etc.,) the client should be aware of all test 
scores and personal records whld' are Input into the sysrtm. 

o The delivery system should be available and accessible to 
adequately serve the user site. 

o Costs for the delivery system components should be at a 
level per user which will make it feasible for user 
agencies or local or state governments to support them 
financially (ACSCI,1981) 

K'e esrllest developed computerized career Information 
delivery systems operated on mainframe computers. Later, some of 
these same systems along with some newly designed systems 
operated on a minicomputer. Now, with the advent of micro- 
computer technology, and with many systems being available for 
operation on the microcomputer, there are several types of 
computerized delivery for systems to be considered. 

At a 1981 conference on "Technology for Career Information 
Delivery" the advantages of the microcomputer were discussed by 
several presenters. Williams pointed out that the most obvious 
advantage of the microcomputer Is cost. 

It Is considerably cheaper to go wld' a micro computer 
version [of a state-wide system) that Involves the local's 
buying and accessing their own equipment rather than running 
through a central mainframe that requires administrative 
support.... There Is also thg advantage of convenience [to 
the user] since there are many programs that can be used In 
addition to the [CIDS] software program, which make the 
computer more cost-effective than the mainframe (Williams, 

At the same conference, Belsse stated that the microcomputer 
offers several advantages over other technologies. 

The Initial cost of a floppy disk system is competitive with 
the cost to an agency of the annul lease of a computer 
terminal and the costs of accessing [the CIDS] on a time- 
shared minicomputer. Where a time-shared computer is 
accessed over long-distance telephone lines, or where 
computer charge rates are high, stand-alone microcomputers 
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can offer substantial economies, especially over a three- or 
four-year period. 

He did point out, however, that "microcomputers are not 
wltl'out their drawbacks," one of which Involve the cost of 
hardware for a system which requires hard disk capability to 
operate (Beisse, 1981.) 

In looking at the needs of the State of New York, it is 
certainly essential to determine the best and most cost-effective 
delivery mode for a computerized career information delivery 
system. 

Although the next topics are not a function of the 
computerization of information components, they are indeed 
crucial to the initial and continuing successful use of a 
computerized -system in any setting. Their importance warrants 
their being mentioned here. 

System support services and training ate important 

components of a system if the information in that system is to be 

of maximum benefits to the clients of a school or agency. 

Ti'ompson and L .T^^ (1985) identified the problem of 

counselors' not using computerized systems because they are not 

familiar with their use. They concluded: 

■B'us, in-service training that emphasizes the variety and 
strengths of the various information files, particularly the 
newly developed local files, and that provides counselors 
with creative strategies for Incorporating the system into 
their counseling efforts, is necessary for the establishment 
and effective functioning of a comprehensive career 
information system. 

In discussing the information competency required of 
counselors in the National Vocational Guidance Standards, 
Whitfield (1985) stated that "Counselors and counselor trainers 
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need to know more about career information and career guidance." 
He further stated that two of the sub-areas of the information 
competency deal with career information and career information 
delivery systems. They are: 

1. Knowledge of education, training, employment trends, 
labor market, and career resources that provide 
information about job tasks, functions, salaries, 
requirements, and future outlooks related to broad 
fields. 

6. Knowledge of strategies to store, retrieve, and 
disseminate vocational/career information. 

Perlmutter (Bloch) and Maze in their chapter on implementing 
career information systems in the book Microcomputers and the 
Sc^'ool Counselor edited by Johnson (1983) stress the importance 
of initial and ongoing training for counselors and aides to 
counselors wt'o will be using the system. In planning for system 
implementation, they urge that resources be allocated both for 
initial and on-going staff development. 

Ti'erefore, the availability and quality of initial and on- 
going training for counselors and support staff on the 
information components along with the use and uses of a 
computerized career information delivery system must be 
considered as an important part of a system. 
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Summary 

Information Components and Design Considerations 
for a Computerized Career Information Delivery System 

Information components are included in the summary of this 
review if they meet one or more of the following criteria: 

-the data element is required to meet the needs of the New 
York curricula for "Home and Career Skills" (for students in 
grades seven and eight); "Introduction To Technology" (for 
students in grades seven and eight); and "Introduction To 
Occupations" (for students in grades nine and ten); 
-the Information component is required to fulfill the 
requirements of the Commissioner's Regulation 100.2 ( j) for 
district guidance plans; and/or 

-the information component is recommended in relevant 
literature for use in the guidance process, with special 
consideration given to populations with special needs, such 
as unemployed adults and people with disabilities. 
During the review of the ll^^rature, it became evident that 
some structure for presenting the findings related to information 
components and system design considerations was needed. In some 
instances the information and system needs expressed by the 
authors were not discussed in terms of categories. Some authors 
were not specific in their definitions of information components, 
and others categorized the information differently. Several 
authors implied that certain types of information was needed, but 
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did not name them specifically. Since it was necessary to impos 
a structure on this variety of information, the broadest possibl 
categories for a computerized system were identified. In all 
cases, responsibility for the decision to include or not to 
include information components or design considerations rests 
solely with the authors of this study. 

I. OCCUPATIONAL INFORMATION 
A^ Information Components Related to Specific Occupations 

1. Occupational title with identifying code 

2. Occupational description 

3. Occupational group or cluster 

4. Job duties 

5. Interests (of workers as validated in various studies) 

6. Aptitudes 

7. Temperaments 

8. Entry level skills required for job 

9. Physical demands 

(amount of weight the worker must lift or carry) 

10. Physical activities 

11. Tools, equipment, etc. used on the job 

12. Environmental /work conditions 

13. Hours of work and travel required on the job 

14. Industries in which occupations occur (places of employment) 
! 5. Hiring channels 

16. Career ladders and advancement opportunities 

17. Occupational outlook 

18. Current and future occupational supply and demand 
(numbers for each) 
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19. Current employment (numbers, by occupation by industry) 

20. Earnings and benefits 

(beginning, average, range, fringe benefits) 

21. Education/training required 

22. Licensing, certification, tests, etc. required 

23. Helpful secondary sd'ool subjects 

24. Related occupations 

25. Related military occupations 

26. Sources of additional information 

Information Ccmponents Related to Laboi Market Entry 

27. Job seeking skills 

28. Self-employment and entrepreneural information 

II. EDUCATIONAL AND TRAINING INFORMATION 

A^ Information Components Related to Secondary Areas of 
concentration 

29. Area of concentration description 

30. Area of concentration group or cluster 

31. Typical course work 

32. Types of learning activities 

33. Skills and knowledge acquired 
34 delated entry level occupations 

35. Related occupations requiring further study 

36. Related post-secondary programs 

Information Components Related to P ost-Seco ndary 
gducatlonai and training frograins 

37. Program description 

38. Program group or cluster 
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39, Typical cr;i se work 

Types icarnlng activities 

41. Skills and knowledge acquired 

42. Certificates or degrees earned 

43. Normal length of time to complete certificate or degree 

44. Related occupations 

45. Related programs of study 

46. Type(s) of Institutions offering program 

£i Information Corogonents Related to Education al and Training 
InstrtutXons " — °- 

47. Name and address of educational or training Institution 

48. Geographic location 

49. Type of Institution 

50. Admission Requirements 

(test scores, application pr .«ec. res, application deadline, 
fees, physical exam, etc.) 

51. Name and address of admissions contact person 

52. Cost (in-state and out-of state tuition, room, board, fees) 

53. Housing 

(availability, on-campus, off -campus, freshmen requirement 
percent of undergraduates living on campus, etc.) 

54. Financial aid inforajation 

(sources, application procedures, application deadline, 
average award) 

55. Name and address of financial aid contact person 

56. Education and training programs offered 

57. Types of certificates or degrees awarded 

58. Normal length of time to complete certificates or degrees offered 

59. Special programs available 

i^S^P®'*^^Y® e<lucation, adult basic education, foreign study. 
ROTC, etc.) ' 
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60. Library and learning center facilities information 

61. Support services available 

(counseling, learning lab, tutoring, placement) 

62. Services for handicapped 

U'ousing and building access, designated parking, counseling, 
interpreters for the deaf, learning aids, registration 
assistance, tutors, transportation for the mobility impaired) 

63. Intercollegiate sports offered 

(by sex and availability of scholarships) 

64. Accreditation information 

Information Components Related to Major Sources of Financial 
Aid and Scholarships 

65. Name 

66. Eligibility requirements 

67. Application procedures and deadline 

68. Size or basis of award 

69. Name and address of contact person 
7". How to obtain additional information 
III. SYSTEM DESIGN CONSIDERATIONS 

71. Interactive structured search of occupations by such eli^ments as 
interests, temperaments, values, skills previously acq. red, etc. 

72. Ability for users to query the system as to why specific 
occupations did not appear on their lists 

73. Direct access to information about occupations 

74. Direct access to information about secondary areas of 
concentration 

75. Direct access to information about post-secondary educational 
and training programs 

76. Interactive structured search of educational and training 
institutions * 

77. Direct access to information about educations alninp 
institutions *^ 
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78. Ability to match institutions and educational programs 

79. Links to published assessments 

80. Ability to use occupational characteristics to sort for 
similar occupations 

81. Interactive structured search of financial aid information 

82. Ability of the system to operate on a microcomputer 

83. Ability of clients to walk away from a session at the computer 
wit* a printed record of searches and information received 

84. Ability to add state specific occupational and educational 
information to existing records 

85. Ability to create records for new and emerging occupations 
and training programs 
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Chapter Three 
METHODOLOGY 



Introduction 

In this chapter, the methodology for the evaluation of 
computerized career information delivery systems will be 
detailed. The study sought to answer several related questions. 
The first question was: "What is needed ir a computerized system 
to meet the career information needs identified for New York 
State?" The second question was: "What systems are already being 
used in the State, how effective and well liked are they by their 
users, and what gaps are there in those systems?" The bird was: 
"What can vendors of systems offer New York State to help meet 
its needs?" Fourth: "How much will that cost the State, 
comparatively?" The final, and ultimate, question was: "Giv-en 
the answers to the four questions, what steps should the State 
take to adopt or develop one or more career information delivery 
systems to provide national and State information to the broad- 
based user community?" 

To answer these questions, five major activities or sub- 
studies were carried out. They were: 

1. The Information Validation Survey. Information 
components and design considerations which were identified 
through the review of the literature were presented in a 
survey instrument with a Likert-type scale ranging from 
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"highly desirable" to "highly undesirable." These 
instruments were sent to a national panel of experts in 
career guidance and a state panel of experts in guidance 
and occupational education. 

2. The Surveys of User Satisfaction. Surveys to determine 
the degree of satisfaction with systems in use as well as 
the overall needs and uses for a system were constructed for 
principals, counselors, students and parents. A selection 
of schools throughout the State using CIS, GIS, CHOICES, 
Discover, SIGI, and C-LECT were surveyed. 

3. Site visits. Selected schools and agencies using the 
major systems were visited to obtain information about the 
uses of and satisfaction with systems through in-depth 
interviews with counselors in those sites. 

4. Request for Information from vendors. Based on the 
results of the Information Validation Survey, the Surveys of 
User Satisfaction, and the site visits, a request for 
information regarding system components and costs was 
developed and sent t-o the vendors identified «»s active in 
New York State. 

5. Demonstration of CiDS to the System Rev^ P. nel. The 
same vendors were invited to make a structured presentation 
to a panel of guidance, oc pational education, 
rehabilitation, and labor pvofessionals both from State 
government and local agencies and schools. A rating 
Instrument was designed witii a Likert-type scale for the 
System Review Panel's evaluation of the systems. 
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each of the above, details of instrumentation, 
population, and analysis will presented in this chapter. 

Information Validation Survey 

The purpose of the Information Validation Survey was to 
ascertain the extent to which the information components and 
design considerations identified in the review of the literature 
were valid. 
INSTRUMENT 

A review of the literature was conducted to identify the 
information components and system design considerations which 
would meet the needs of the implementation of the New York State 
occupational education curricula, "Home and Career Skills" 
"Introduction to Technology", and "Introduction to Occupations," 
the implftsientation of the guidance plans, and the needs of the 
various populations making career decisions. The review included 
an analysis of the curricula, the guidance plan guidelines, 
standards promulgated by professional organizations, and other 
writings. 

From thit review, 85 separate elements were identified. For 
ease of understanding, they verri grouped under three major 
headings: occupational information, educational and training 
information, and system design considerations. Each of the&e 
major headings ws« further sub-divided into appropriate topics. 
Under occupational information, the headings included information 
components reltced to specific occupations, such as job duties. 
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entry level skills required for the job, occupational outlook, 
earnings and benefits, and information components related to 
labor market entry. 

Under educational and training information, the first sub- 
component related to secondary areas of concentration, including 
such items as typical course work and related entry-level 
occupations. Ths second sub-heading related to post-secondary 
educational and training programs and had similar information 
items. The third sub-heading related to educational and training 
institutions, and the fourth to sources of financial aid and 
scholarships. 

The third major section was system design considerations and 

included items related to the structure, rather than the 

content., of a career information delivery system. 

Since the respondents were asked to evaluate the items in 

relation to new .occupational education curricula and the 
district gv-^c^xi<^-- piins, excerpts from these documents were 

included wifcrt t he «i rvey. 

The respondents were asked to rate each item as highly 
desirable, desirable, neither desirable nor undesirable, 
undesirable, or highly undesirable by circling the appropriate 
number. Respondents were also given the opportunity to conaient 
on or make additions to frry of the items on the sur\v»y. The 
Information Validation Survey, the excerpts from the curricula 
and guidance plan guidelines, and relateJ cover letters are 
attached as Appendix I. 



ERIC 



86 



CIDS Evaluation, 74 



POPULATION 

The survey was mailad to a national panel of experts and a 
state panel of guidance and occupational education professionals. 

The national panel was selected frc.n those who have made 
major contributions to the field of career development and have 
been active in the major organizations related to career 
development including the Association of Computer-Based Systems 
for Career Information, the National Career Development 
Association, .».ivj le Aiaerican Vocational Association — Guidance 
Division. Soin^ of their many accomplishments are descr bed 
below. Each has won many awards and has published extensively 
in the field. 

The national panel consisted of the following people: 
-Dr. Norman Gysbers, ct'rrently professor of educational and 
counseling psychology at the University of Missouri; 
previously president of the National Career Development 
Association and the American Association for Counseling 
Development, and editor of the Vocational Gui dance Quarterly 
and The Journal of Career Development . 

-Dr. L. Sunny klansen, currently professor of counseling and 
student personnel psychology at 'jhe University of Minnesota 
and president of the National Career Development 
Association; previously editor of The School Counselor and a 
Fulbright Scholar. 

-Dr. Edwin L. Herr, currently professor and head of the 
Division of Counseling and Educational Psychology at 
Pennsylvania State University; previously president of the 
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American Association of Counseling and Development, National 
Career Development Association, and the Association for 
Counselor Education and Supervision, and editor of several 
professional journals. 

-Dr. Kenneth Hoyt, currently University Distinguished 
Professor at Kansas State University; previously the 
Director of the Division of Career Education, U.S. 
Department of Education, president American Association 
for Counseling and Development, and editor of Counselor 
Education and Supervision . 

-Dr. Cynthia S. Johnson, currently associate professor. 
Department of Higher and Adult Education, Teachers College, 
Columbia University; previously president Amerlv-an College 
Personnel Association, member of the editorial board of 
Career Planning and Adult Development J ournal , and principal 
investigator of research on the career decision making of 
women and minorities funded by the W. W. Kellogg Foundation. 
-Carol J. Kososki, currently director. South Carolina 
Occupational Information System, president of the 
Association of Computer-Based Systems for Career 
Infcrmation, and winner of the National Occupational 
Info, ition Coordinating Committee Direct Award for 
Outstanding Contributions for the If emenc of 
Occupational Education in the United States. 
-Dr. Evelyn J. Lavaty, currently coordinator of Career 
Guidance and Counseling, director of Rural Education, 
Nebraska Department of Education, chair of the State 
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Supervisors of Career Guidance Network, mcnber of the 
American Vocational Association — Guidance Division Policy 
Committee, and volunteer occupational therapist in a 
Veteran's Hospital. 

-Dr. David B. Tiedeman, currently president of the 
International College and vice president of the 
LIFECENTER Foundation; previously professor of Education at 
the University of Southern California, Northern Illinois 
University, and Harvard, director of Project Ti-iENT, 
president of National Career Developnient Association, 
Division of Counseling of the American Psychological 
Association, and National Council of Measureient in 
Education. 
STATE PANEL 

A state panel was sought to respond to the Information 
Validation Survey that would include the major groups interested 
in the final product — a computer-based career information 
delivery system — and that would have knowledge of the kinds of 
career infornation needed by the groups to be seived. 

The Information Validation Survey was sent to 122 people, 
incl -Uing: 

-67 major providers of occupational education (who were 
asked to h:*^^ the survey completed by "someone in your 
organization who is familiar with the career information 
needs of the people you serve. This could be a director of 
guidance, a director of occupational education programs, a 
counselor, an occupational education teacher, or an 
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administratoi:;'' ) 

-25 teachers and State Education Department p o 5ir ^->?l3 
involved in the development of the occupational «ut cr i 
curricula; 

-10 professionals identified by the Office of Vocacicnai 
Rehabilitation; 

-10 field professionals identified by the Office of 
Employment and Training, Job Training Partnership Act: 
-5 professionals associated with Education Information 
Centers ; 

-3 professionals recommended by the Commission for the Blind 
and Visually Handicapped; 
-1 Director of MetroGuide; and 
-1 Director of Adult and Continuing Education. 
ANALYSIS 

The data were analyzed using LOTUS 123 (Lotus Development 
Corporation, 1985) andSTATS-2, Release 2.0 STATSOFT, 1985). 
For each item, a mean and standard deviation were computed 
separately for the national and state panels. In the analysis, 
unansweied items or items with more than one response wers 
handled as missing data and eliminated from the analysis. 

Comments on the survey responses were read and incorporated 
In the findings. 

Surveys of User Satisfaction 
The purpose of the Surveys of Ui',sr Satisfaction was to 
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determine the degree to which various groups were satisfied with 
the computer-based caref-r information delivery systems they were 
using as well as what each of these groups would like to see in 
career information deliver systems in the future. The four 
groups were principals, counselors, students, and parents. 
Ii-ISTRUMENT 

Because the needs, interests, and points of view of the four 
groups differ from each other, four different instruments were 
designed for this survey. Tie content of each of the instruments 
was based, again, on the information components and design 
considerations identified in the review of the literature. Each 
instrument is described below, and all of the Surveys of User 
Satisfaction ^nd related cover letters are included as Appendix 
II. 

The Principal's Survey of User Satisfaction 

TiVe Principal's Survey of User Satisfaction contained twenty 
items divided into two sections. The first section consisted of 
ten items which asked how the school had used their CIDS to 
support various guidance and educational activities. Principals 
were asked to respond "yes" or "no" to such activities as: 
"irr.A ■ j.ixl career counseling," and "occupational education 
progr&ffl6." TJ'e second section consisted of ten statements for 
each of which the principals were asked to indicate their level 
of agreement or disagreement using a Likert-type scale- One of 
the items, for example, was: "The information presented in the 
CIDS is appropriate for the students in my school." Provision 
was made for additional comments. 
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The Counselor's Survey of User Satisfaction 

The Counselor's Survey of User Satisfaction consisted of 
seven sections. The first section was similar to the pritvcipal's 
and asked the counselors to identify activities for which the 
CIDS had been used in their school. Ti'e second section asked the 
counselors to respond "yes" or "no" to possible individual 
student outcomes they had observed from the use of CIDS, such as, 
"increased awareness of self in relation to occupational and 
educational choices." Using a Likert-type scale t'us - -..nselors 
were asked to r : :e the usefulness of 20 of the infc v.'.i >n 
components, design considerations, and related mate. . ' such as 
handbooks and training. The fourth section dealt with training 
received and its usefulness. TA'e fifth section addressed 
implementation guides. The sixth section dealt with user session 
time, and the last section dealt with the user friendliness of 
the software and the presentation of the information. 
Opportunity for comments was provided in the last question which 
asked, "Tf you were designing your ideal c».nputerized career 
information delivery system, what would be its most important 
features?" 

TJ'e Student's Survey of User Satisfaction 

The Student's Survey c User Satisfaction consisted of 15 
items calling for a "yes" or "no" response. They ranged from 
questions about the ease of system use to the outcomes of having 
used the CIDS. Students were also asked to specify their grade 
in high school, the system they had used, and the number of times 
they had used it. Again, provision for comments was made through 
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an open-ended question: "If you have any suggestions to help 
make a computerized system more useful, please write them on the 
back of this sheet." 

Parent's Survey ^1 User Satisfaction 
The Parent's .S.-.-vey of User Satisfaction was divided into 
two sections. Tht 3t section was designed to provide 
information about the parent's opinion of the quality of the 
career information systenn at the student's school. The second 
section asked the parents whether or not they believed a computer 
would be a good tool to carry out ten specific guidance purposes, 
such as "show relationships between work and educational 
programs." ^ 
POPULATION 

Since the purpose of the Surveys of User Satisfaction was to 
determine the level of satisfaction of a varied pool of public 
secondary school users, sites were selected from as many as 
possible of the vendors of CIDS. At the same time, an attempt 
was made to have adequate geographic coverage of the State. 
However, there was no central, reliable source of the schools 
using at ^ or all c" the systems. Therefore, schools were 
identifier throug'. * variety of means. The details of selection 
of the «>i sci'ools surveyed are as follows: 

-For CIS, 15 high schools using HetroGuide were identified 

by the New York City Board of Education; 

-For GI3, 15 schools were selected from a state-wide list 

provided by the Onondaga-Madison BOCES ; 

-For CHOICES, five schools were selected f'om a list provided 
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by the vendor; 

-For C-Lect and SIGI, one school each was surveyed from a 
list supplied by the vendor. In both cases, they were the 
only secondary schools on the vendor's list; 
-For Discover, four secondary schools were selected from a 
list supplied by the vendor. It should be noted that this 
is far fewer than originally planned, but delays in 
receiving the list of sites and lack of information on the 
number of sites contributed to the number selected. In 
fact, the deadline for return of survey instruments was 
extended to be able to include any Discover sites. 
-COIN was not included in the Survey of User Satisfaction 
because, according to information supplied by the vendor, 
there were no secondary schools using their computerized 
system in the State. 

Each school selected was sent a package consisting of th. 
following surveys: 

-one Principal's Sur ey of User Satisfaction; 

-one Counselor's Survey of User Satisfaction; 

-10 Student's Survey? User Sstisf action; and 

-10 Parent's Surveys o-. iJssx- ^.iaCisf action. 

Separate cover letters were written to each of the above 
groups and provided in like quantity. In addition, stamped 
return envelopes were provided for the students and for the 
parents. The principals were asi--v to complete their surveys, to 
give the counselor's survey to the person most lamiliar with the 
CIDS in the school, to give the 10 student surveys to students 
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who had used the CIDS, and to give these students the parent 
surveys to take home as well. Principals, counselors and parents 
were also given copies of a letter written by Assistant 
Commissioner James Kadamus authorizing the study and requesting 
their cooperation. Copies of these letters are included with the 
surveys in Appendix II. To encourage returns, principals were 
asked to be responsible only for the distribution, not the 
collection, of the surveys. 

Particular assistance in assuring returns wa ceired from 
Larry Edwards, Director of Pupil Personnel Scrvicco «t the New 
York City Board of Education, who saw to it that the appropriate 
memorandum from the Executive Director of the High School 
Division was sent to the principals being asked to complete the 
survey. Similarly, Robert Joyce, Manager of Instructional 
Computing at the Central New York Regional Co-nputer Services 
Center, used his electronic network to inform other computer 
center managers of the survey and to ask their cooperation. It 
should be noted that when the principals returned their surveys, 
there was a generally good response from the others in their 
schools. Special thanks go to those principals, and th«ir names 
are listed with acknowledgment in Appendix III. 
ANALYSIS 

The data were entered and analysed using LOTUS 123 (Lotus 
Development Corporation, 1185) and STATS 2, Release 2.0 
(STATSOFT, 1985.) The original intent for analysis had been to 
look at the data aggregated across all systems within each survey 
and then to compare the results by system for eac'a survey. There 
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were too few respondents to carry out this plan completely. 
Therefore fewer comparative analyses were made. The description 
of the analyses are presented below for each of the surveys. 
The Principal's Survey of User Satisfaction 

For the first section, the percentage of principals responding 
positively to each of the possible uses of CIDS in their schools 
was comput.2d. For the second section, the mean and standard 
deviation of the level of satisfaction with their CIDS was 
computed for each item. In addition to the aggregate analysis, 
a comparison using a t-test of responses from principals of 
schools using CIS was made with those from principals of schools 
using GIS. 

The Cour.^ a lor's Purvey of User Satisfaction 

For the first twenty items, the percentage of counselors 
responding positively to each of the possible uses of CIDS and 
outcomes from CIDS in their schools was computed. For the 20 
questions on Information components and related materials as well 
as for the questions on tral Ing and handbooks, the mean end 
standard deviation for assessment of usefulness on the five point 
scale was computed. The mean was computed for counselors' 
estimates of average time per session and desirable time per 
session at the computer. 

Using t- tests, comparisons were made between the responses 
of counselois from schools using CIS with those of counselors 
from schools using GIS. 

Additional comments on the response sheets wera read and 
recorded for the findings. 
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The Student ' s Survey of User Satisfaction 

The student's survey consisted of 15 question*. For each 
Item, the percentage of students responding positively was 
computed. 

Using analysis of variance, a comparison was made between 
the means of students from schools providing CIS, those from 
schools providing GIS, end those from schools providing CHOICES. 

In addition, analysis of variance was used to examine 
differences in responses by grade levels, aggregating all systems. 
The Parent's Survey of User Satisfaction 

The percentage of parents responding positively to each 
question was computed. 

Only the first five questions on the parent survey were 
designed to compare systems. For these questions, analysis of 
variance was used to compare the responses of parents whose 
children attend schools providing CIS with those providing GIS 
and those providing CHOICES. The remaining questions were 
designed to elicit parents' opinions of what a computerized 
system should do and were therefore not subjected to comparative 
analysis. 

The Site Visits 

The purpose of the site visits was to examine the systems 
in schools and agencies where they were in use and to gain the 
opinions of counselors in a less formal manner so that issues 
which might not emerge in a survey would surface. In addition. 
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where the Surveys of User Satisfaction dealt only with secondary 
school sites, the visits were made to a variety of types of 
locations. 
INSTRUMENT 

A questionnaire was devised that centered on four key 
questions: 

-the major programmatic goals of the career counseling 
program; 

-the most important kinds of information needed by clients 
to make career and educational decisions; 
-how a computer-based CIDS can help meet the goals and 
deliver information; and 

-the major strengths and weaknesses of the system in use. 
A copy of the form used by the site Iricerviewer is attached as 
Appendix IV. 
POPULATION 

Visits were made to five high schools, one junior college, 
and one local vocational rehabilitation office. 
ANALYSIS 

The information obtained during the site visits was recorded 
on the forms at the time of the interviews. The responses were 
examined to see how well the systems in use were meeting the 
programmatic goals of the career counseling program and the 
career decision needs of individuals. Results were summarized by 
type of institution visited and across all types. 
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THE REQUEST FOR INFORMATION FROM VENDORS 

The purpose of the request for information from vendors 
(RFI) was was to provide a basis for assessing the contents, 
benefits, and costs of each system and to compare them with each 
other on criteria that were as specific as possible. 
INSTRUiHENT 

The first four sections of the RFT were based on the 
information components and design considerations identified in 
the review of the literature and confirmed hy the Information 
Validation Survey, the Surveys of User Satisfaction and the site 
visits. It was composed of 99 items concerning occupational and 
educational information. For each item, the responding vendor 
was asked to state whether the item was currently In the system 
at the national level of analysis, whether it was in the system 
at the New York State level of analysis, and whether or not the 
system had the capability of incorporating the information at the 
New York State level. Following that, the vendor was asked to 
indicate whether or not the system incorporated particular design 
features, materials and training. 

In the next four sections, the vendors were asked to provide 
information concerning the length of user sessions, the machines 
and media available, the availability of research and evaluation 
studies concerning the use of their system as a state-based CIDS, 
and any ^idditional or optional features of the system. 

Vendors were also asked to provide printouts of a particular 
occupations, program of study, and educational institution to be 
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used in assessing the comprehensiveness and accuracy of their 
data. 

Finally vendors were asked for the costs of their system. 
Since all vendors price their systems in different ways based on 
many variables that they determine, a pricing scenario was 
developed so that costs could be compared on a similar basis. 
The scenario enabled the investigators to deal with a number of 
issues of interest to the State, including entry of State 
developed data and files into each of the systems. 
The scenario is presented below: 
Assume the following: 

A. Your system is a New York State career information delivery 
system. 

B. You are delivering your basic system of occupational and 
educational information without any additions of data or files. 

C. In the first year of delivery, there are 100 MS-DOS (IBM 
compatible) junior and senior high school sites; in the second 
year, 100 similar sites are added; and in the third year, you are 
delivering to a total of 300 sites. 

1. What is the total cost of the system for each of the 
three years? Provide a work sheet to show the breakdown of costs 
for such items as initial and subsequent license fees to the 
State; per site license fees; other software fees; training of 
counselors; counselor implementation handbooks @ 1 per site per 
year; indices to the system @ 2 per site per year; user handbooks 
@ 500 per site per year; and any other costs needed so that the 
system can be fully implemented, delivered and used. 
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2. If the State provided data on entry level wages, 
employment outlook, and licensing requirements for 600 SOC based 
occupations, what would be the total costs for each of the three 
years to enter and maintain that information in your system. 
Provide a breakdown of these costs by data entry costs to the 
State; by site use costs and/or other costs as applicable to the 
practices of your organization. 

3. If the State wanted to enter the data in your system, 
would you provide the documentation necessary? What would be the 
cost of this option for each of the three years? Provide a 
breakdown as in 2. 

Assume the State created an entirely new file which listed 
350 JTPA program providers with information on the area and 
clients served and with links to the relevant occupational 
descriptions in your file. 

4. What would be the total cost to the state for your 
system to incorporate and deliver this file over the the 

three years described above? Provide a breakdown as requested in 
question 2. 

5. Describe your policies concerning copyrights and use of 
state modifications to your system. 

6. If you have available other systems in addition to the 
basic system and modifications for which you provided costs in 
answer to the above questions, please describe those systems and 
the cost of utilizing them in the three-year scenario described. 

N.B.: Please be sure that all costs provided are accurate 
and realistic. While this is not a formal bid, it is expected 
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that costs over the next three years will be consistent with 
those provided herein. 

The Request for Information is attached as Appendix V. 
POPULATION 

The RFI was sent to the six vendors who had been identified 
as major providers of CIDS in New York State. They were C-Lect, 
CHOICES, CIS, Discover, GIS, and SIGI. 
ANALYSIS 

The data were entered and analyzed using LOTUS 123 (Lotus 
Development Corporation, 1985) and STATS-2, Release 2.0 
(STATSOFT, 1985) as described below for each section of the liFI. 
Information Components , System Design, and Support Materials 

A point system was established for the items related to 
information components and system design features. In this 
system, items were assigned a value of one, two, or three points, 
based on the results of the previous surveys. In addition, some 
information components were only counted for netional information 
while others were counted for national and state information. 
This decision was based on information provided by the office of 
Research and Statistics, New York State Department of Labor. The 
total number of points possible was 300. 

A comparative analysis of the percentage of points for all 
information components, design considerations, and support 
materials were carried out, as well as separate comparisons for 
national information available. New York State information 
available, the ability to add Kew York State Information, system 
design considerations, and user materials and support. Details 
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of the number of points assigned to each information item are 
presented in Appendix VII, Procedure and Instruments for 
Evaluating CIDS. 
Length of Session 

A comparison was made of the vendors' estimates of the 
average length of time it generally takes high school students 
to complete a session at the computer in which they use the 
occupational exploration strategy or strategies and generate a 
list of occupations meaningful to them. 
Machines and Media 

A chart was developed to present the comparative 
capabilities of the systems to run on different time share and 
microcomputers . 

Co m pr e hens ivene s s of Information 

The printouts of occupational and educational institution 
information provided by each vendor were also analyzed. The 
occupational information printouts were analyzed by a former 
SOICC Director and CIDS Manager and the educational information 
by the New York State Office of Higher and Professional 
Education. There were 20 items measurable for comprehensiveness 
and accuracy on a scale of 5 to 1 ranging from "excellent" to 
"atrocious." If a system was completely lacking in information 
on any item, that item was scored 0. The highest possible score 
for each system was 100. Each system's total score was computed 
and a chart comparing the per item scores and total scores was 
created. Details for scoring are presented in Appendix VII, 
Procedure and Instruments for Evaluating CIDS. 
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The items which were scored are: 
Occupational Information 

bias-fr<v3 jot titles and descriptions 

occupational description 

job duties, including tools and equipment used, if 

applicable 
interests and temperaments 
aptitudes 

skills, education, and training required for entry level 

positions 
physical demands and activities 

working conditions (includes environmental conditions 

and hours of work and travel) 
future outlook 
earnings 

related occupations, including occupational group 
places of employment 
Educational Institution Information 
admissions requirements 

cost (in^state tuition, room and board for year 

specified by system) 
type of institution 
housing availability 

information about graduate ( post-baccalaursate) 
programs in education 
services for people with handicapping conditions 
enrollment (number of students) 
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intercollegiate sports (not able to be rated by the 
group rating the infoi nation in this study) 
The following sources were used to validate the occupational 
information: 

-SOC Career Profiles (National Crosswalk Center, 1985;) 
- Occupational Projections and Training Data (U.S. Department 
of Labor, 1986; ) 

- Occupational Outlook Handbook , 1986-87 (U.S. Department of 
Labor, 1986;) and 

- Guide for Occupational Exploration (U.S. Department of 
Labor, 1984.) 

Costs 

The vendors' presentations of costs were compared in the 
following ways: 

1. the cost of implementing the system in each of three 
years in the scenario described earlier in this section of 
the chapter under "Instrument." 

2. a three-year cost analysis of a vendor maintained State 
database; 

3. a three-year cost analysis of a State maintained State 
database; and 

4. a three-year cost analysis of a vendor developed file of 
State JTPA program providers as described in the scenario. 

Other Information in the RFI 

Other information obtained from the RFI such as vendor 
policies concerning copyrights, research on their system, and 
cptional features was recorded. 
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Demons t rat ioti of CIDS to the System Review Panel 

Ti'e purpose of demonstrating the CIDS to the System Review 
Panel was to allow state users to evaluate the appropriateness 
and useability of the systems with the populations they or their 
agencies serve. Two areas of particular concern were the use of 
systems in classtoom or group settings at the junior or senior 
high school level and the possible contribution of the systems 
to an individual's career planning. 
INSTRUMENT 

A rating booklet was developed for use by each 
participating review panel member. The introduction to the 
booklet presented a brief history of the project, the goals of 
the system demonstration review, an agenda for the day and 
directions. Following the introductory pages, there were six 
differently colored sets of pages to be used for taking notes and 
rating each system. 

Tie actual rating sheets consisted of 15 items to be marked 
"excellent" to "atrocious" using a five point Likert-type scale. 
Three criteria were used to select the items. First, they had to 
be important to users as determinsd through the previous work of 
this study. Second, they had to be system features which could 
be observed in the course of the demonstration. Finally, they 
had to be non -duplicative of information gathered through the 
RFI. The introduction to the instrument and one copy of the 
notes and rating pages are attached as Appendix VI. 
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POPULATION 

There were two populations at the demonstrations. The first 
was the six vendors, all of whom had responded to the RFI. They 
were: C-LECT, CHOICES, CIS, Discover, CIS, and SIGI. Second, was 
the panel invited to review the demonstration. 

In forming the review panel, the goal was to have 
representation from as many user groups as possible, from all the 
geographic areas of the State, and from both counseling 
practitioners and administrators. Accordingly 50 people were 
invited. Hey were drawn from the following groups and from 
suggestions made by these groups: the SOICC Statutory Committee, 
T/'e SOICC Advisory Committee, the Office of Occupational and 
Continuing Education of the State Education Department, the major 
providers of occupational education across the State, the Office 
of Higher and Professional Education, the Education Information 
Centers, the Office of Vocational Rehabilitation, and the 
Department of Labor. 

It was made clear to all potential panel members that they 
would have to participate in the entire day's proceedings so that 
they could produce ratings that would be comparable for all 
systems. The letters of invitation and response forms are 
attached as part of Appendix VI. 
PROCEDURE 

Prior to the day of the presentation and demonstration, June 
24, 1986, the vendors had been informed of the format of the day 
and their related responsibilities. In addition, a meeting of 
the vendors was held the evening prior to the demonstration to 
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insure that all questions regarding purpose and procedure would 
be answered, and to determine the order of morning presentations. 

On the morning of June 24th, the System Review Panel was 
introduced to the purpose of the demonstration and their role. 
The idea of rating, rather than ranking, systems was stressed. 
It was made clear to them that all systems could be rated high, 
or low, on any or all of the criteria. 

Following the introduction, each vendor spoke for 15 minutes 
on the four pre~assigned topics: 

-an overview of the system; 

-the philosophy underlying the system; 

-the use of the system in classroom or group settings at the 
junior or senior high school level; and 

-the major contribution the system makes to individual's 
career planning. 

No representative from any other CIDS was present during any 
of the oral presentations to avoid the problem of rebuttal. Each 
CIDS representative spoke only of his or her system. Panel 
members had color coded sheets in their rating books for taking 
notes on the speeches so that the knowledge gained from them 
could be incorporated in their ratings. 

In the afternoon, the CIDS were demonstrated to the System 
Review Panel by their representatives. Tables witi' appropriate 
electrical connections were placed around the perimeter of a 
large room. Ead' CIDS was assigned a table. The vendors had 
been instructed to bring their own equipment as well as counselor 
implementation materials and student user handbooks. 
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The members of the System Review Panel had been divided 
arbitrarily into six groups. Each group began at the table of a 
vendor. The CIDS was demonstrated for a half-hour. At the 
conclusion of each ha If -hour, five minutes was allowed for 
completing the ratings and transition. Then the groups moved 
around the room to the next CIDS demonstration. In all each 
vendor presented to each of the groups, and all of the groups 
viewed each CIDS for one half-hour. There was some time at the 
end of the afternoon for panel members to return to any of the 
CIDS representatives to ask for clarification or expansion of any 
points . 

The rating booklets were collected as the members of the 
System Review Panel left the demonstration area. 
ANALYSIS 

Data from the rating instruments were entered and 
analyzed using LOTUS 123 (Lotus Development Corporation, 1985) 
and STATS-2, Release 2.0 (STATSOFT, 1985). 

The mean and standard deviation of the responses for each 
item was computed by system and compared in a chart. 

A rating scale totaling 100 points was created by weighting 
five of the original fifteen items, each of which was worth a 
possible high score of 5. The resulting rating instrument is 
included in Appendix VII, Procedure and Instruments for 
Evaluating CIDS. The system demonstration score for each system 
was then computed and compared in a bar graph. 

Finally, the scores of the CIDS on the RFI, the accuracy and 
comprehensiveness of information, and the System Review Panel 
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ratings were combined to give an overall rating to each system. 

Results of the analysis of the System Review Panel as well 
as the other analyses described are presented in the next 
chapter, "Findings." 
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Chapter Four 
FINDINGS 



Introduction 



In this chapter, the results of each of the surveys and 
information gathering activities are presented along v±ti a 
discussion of the implications of the results to the study in 
general. At the end of the chapter, conclusions drawn from an 
examination of all the information gathered will be presented. 

The major sections of this chapter are: 

-The Information Validation Survey; 

-The Surveys of User Satisfaction; 

-The Site visits; 

-Information from Vendors; 

-Demonstration of CIDS to the System Review Panel; 

-Conclusions; and 

-Final Ratings of the CIDS. 

The Information Validation Survey 

The Information Validation Survey was sent to two panels, a 
panel of national experts and a panel of state experts in the 
uses of career information. Of the eight national experts, 1007. 
responded to the survey. Of the 122 state experts, 72 or 59% 
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responded. 

The purpose of the survey was to validate the information 
components and design considerations identified through the 
review of the literature. Respondents were asked to circle the 
appropriate evaluation for each item. The possible responses 
were: 

5 « Highly Desirable 
4 » Desirable 

3 =« Neither Desirable Nor Undesirable 

2 - Undesirable 

1 - Highly Undesirable 

Examination of the national panel and state panel mean 
ratings for each of the 85 information components and 
system design considerations shows that all items were validated. 
In addition, there was remarkable consistency between the 
national and stfite panels as to which items received high mean 
ratings and which relatively low mean ratings. 

Therefore, in carrying out the subsequent evaluations of 
systems through the requests for information from the CIDS and 
the presentation and demonstration of the CIDS to the System 
Review Panel, all of the information components and design 
considerations were incorporated. Further in calculating the 
ratings associated with the CIDS evaluations, the information 
about the relative rating of items by the national and state 
panels was used to help determine item weights. 

The various information items and design considerations are 
grouped below for ease of discussion. 
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OCCUPATIONAL INFORMATION COMPONENTS 

A closer look at the occupational information components, 
presented in Table 1, Information Validation Survey: 
Occupational Information, Responses of the National and State 
Panels, shows little variation in the comparative ratings of 
these information components. It may be noted that the highest 
mean rating of an item by the national panel was 5.00 and the 
lowest rating was 4.00. No rating, therefore, was below 
"Desirable," and 64% of the items were rated 4.50 or better. The 
highest mean rating of these items by the state panel was 4.92 
and the lowest rating was 3.69. The state panel rated 46% of the 
items 4.50 or better. While the state panel ratings were 
slightly lower overall than the national panel. Table 1 shows 
correspondence between the two panels in the relative worth given 
to each i?;em. 

Please note, that item titles are abbreviated in this table 
and others related to the Information Validation Survey. The 
full titles are in the instrument which can be found in Appendix 
I. All tables related to the Information Validation Survey 
follow the discussion of this survey. 
SECONDARY EDUCATION INFORMATION COMPONENTS 

The mean ratings of the panels to the items describing 
secondary areas of education are given in Table 2, Information 
Validation Survey: Secondary Education Information, Responses of 
the National and State Panels. The mean ratings of all items by 
both panels ranged from 4.13 to 4.75, well above "desirable." 
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POST-SECONDARY EDUCATION INFORMATION COMPONENTS 

The section about post-secondary education information 
components is comprised of both program of study and educational 
institution information. 

The national panel rated 22 of the 28 items at 4.50 or 
higher. Type of institution, admission requirements, programs 
offered, and degrees offered were all rated 5.00. Three of the 
same items, admissions requirements, programs offered, and 
degrees offered also had the highest mean state panel rating. 
Both the national and state panels gave their lowest ratings to 
information about library facilities and intercollegiate sports. 
The means and standard deviations are presented in Table 3, 
Information Validation Survey: Post-Secondary Education 
Information, Responses of the National and State Panels. 
FINANCIAL AID AND SCHOLARSHIP INFORMATION COMPONENTS 

The importance of information about financial aid and 
scholarships in the eyes of both the national and state panel 
members can be seen in the details of Table 4, Information 
Validation Survey: Financial Aid & Scholarship Information, 
Responses of the National and State Panels. The national panel 
gave three of the six items a mean rating of 5.00 with no 
variation by any panel member. All of the state panel ratings 
are above 4.50, except for one, and that one is at 4.44. 
SYSTEM DESIGN CONSIDERATIONS 

This portion of the study was used to validate some system 
features rather than information components. All of the features 
were rated well above 4.00 by both panels except for links to 
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published assessment Instruments which was rated 3.88 by the 
national panel and 3.92 by the state panpl. The means and 
standard deviations are presented in Table 5, Information 
Validation Survey: System Design Considerations, Responses of 
the National and State Panels. 
COMMENTS 

Comments from the respondents generally echoed their support 
of the items listed. One state panel respondent wrot-?, "Whoever 
put this together is to be highly commended. Such a system is 
very desirable and would be most helpful to our students." 
national panel respondents, in addition to commenting positively 
on the list of items, noted that the needs of individuals differ 
with their age and stage of career development. One of the 
National Panel members stressed the need for exploration of such 
areas as work and the family and work environment, areas that 
would bring the occupational education curricula and the career 
information components together. 
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Table 1 

INFORMATION VALIDATION SURVEY: OCCUPATIONAL INFORMATION 
RESPONSES OF THE NATIONAL AND STATE PANELS 



Survey Item National Panel State Panel 







Mean 


SD 


Mean 


SD 


1. 


Occupational title and code. 


4.88 


0.35 


4.65 


0.63 


2. 


Occupational description 


5.00 


0.00 


4.92 


0.28 


3. 


Occupational group or cluster 


4.75 


0.46 


4.47 


0.58 


4. 


Job duties 


4.88 


0.35 


4.88 


0.33 


5. 


Interests of workers 


4.63 


0.52 


4.19 


0.82 


6. 


Aptitudes 


4.63 


0.52 


4.61 


0.64 


7. 


Temperaments 


4.38 


0.52 


4.18 


0.92 


8. 


Entry level skills required for job 


4.88 


0.35 


4.83 


0.38 


9. 


Physical demand^ 


4.38 


0.74 


4.36 


0.68 


10. 


Physical activities 


4.25 


0.46 


4.31 


0.76 


11. 


Tools, equipment, etc. 


4.13 


0.64 


4.29 


0.72 


12. 


Environmental/work conditions 


4.50 


0.53 


4.44 


0.58 


13. 


Hours of work and travel 


4.00 


1.07 


4.22 


0.74 


14. 


Industries 


4.75 


0.46 


4.50 


0.61 


15. 


Hiring channels 


4.13 


0.64 


4.10 


0.72 


16. 


Career ladders 


4.50 


0.53 


4.37 


0.70 


17. 


Occupational outlook 


4.88 


0.35 


4.75 


0.50 


18. 


Supply and demand 


4.50 


0.76 


4.57 


0.58 


19. 


Current employment 


4.25 


0.71 


3.82 


0.82 


20. 


Earnings and benefits 


4.88 


0.35 


4.63 


0.49 


21. 


Education/ training required 


5.00 


0.00 


4.92 


0.28 
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Table 1 Continued 





SuTvev T' Am 


National Panel 


btate 


Panel 






Mean 


SD 


Mean 


SD 


22. 


Licensing, certification 


4.63 


0.52 


4.78 


0.45 


23. 


Helpful secondary school subjects 


4.38 


0.74 


4.53 


0.58 


24. 


Related occupations 


4.63 


0.52 


4.26 


0.65 


25. 


Related military occupations 


4.13 


0.83 


3.69 


0.82 


26. 


Sources of additional information 


4.25 


0.89 


4.38 


0.62 


27. 


Job seeking skills 


4.88 


0.35 


4.53 


0.61 


28. 


Entrepreneurial information 


4.63 


0.52 


4.19 


0.72 
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Table 2 

INFORMATION VALIDATION SURVEY: SECONDARY EDUCATION INFORMATION 
RESPONSES OF THE NATIONAL AND STATE PANELS 





ourvsy item 


National Panel 


State 


Panel 






Mean 


SD 


Mean 


SD 


29. 


Area of concentration description 


4.12 


0.83 


4.53 


0.63 


30. 


Area of concentration cluster 


4.13 


0.83 


4.33 


0.67 


31. 


Typical course work 


4.50 


0.53 


4.42 


0.67 


32. 


Types of learning activities 


4.25 


0.89 


4.29 


0.68 


33. 


Skills and knowledge acquired 


4.75 


0.46 


4.49 


0.71 


34. 


Related entry level occupations 


4.50 


0.53 


4.51 


0.61 


35. 


Related occ's further study 


4.25 


0.71 


4.31 


0.65 


36. 


Related post-secondary programs 


4.38 


0.74 


4.32 


0.65 
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Table 3 

INFORMATION VALIDATION SURVEY: POST-SECONDARY EDUCATION INFORMATION 
RESPONSES OF THE NATIONAL AND STATE PANELS 



Survey Item National Panel State Panel 

Mean SD Mean SD 

(Program Descriptions) 



37. 


Program description 


oo 


U . JD 


/. CO 

4.58 


0.65 


38. 


Program group or cluster 


^ . C J 


U . Dl 


4.31 


0.65 


39. 


Typical course work 


^ . DU 


Urn Do 


A OA 


0.64 


40. 


Types of learning activities 


A 

ZD 


n Q o 


4.21 




41. 


Skills and knowledge acquired 




U. 3 J 


4.52 


0.58 


42. 


Certificates or degrees earned 


/■ Q Q 


0.35 


4. 55 


0.58 


43. 


Length of time to complete 


A IK 


U.4d 


4.51 


0.61 


44. 


Related occupations 


4.50 


0.53 


4. 34 


0.61 


45. 


Related programs of study 


4.50 


0.53 


4.27 


0.60 


46. 


Institution types 


4.88 


0.35 


4.37 


0.62 


(Institution Descriptions) 










47. 


Name/address 


4.88 


0.35 


4.64 


0.61 


48. 


Geographic location 


4.63 


0.52 


4.53 


0.69 


49. 


Type of institution 


5.00 


0.00 


4.49 


0.65 


50. 


Admission Requirements 


5.00 


0.00 


4.65 


0.63 


51. 


Admissions contact 


4.50 


0.53 


4.29 


0.80 


52. 


Cost 


4.88 


0.35 


4.60 


0.60 


53. 


Housing information 


4.50 


0.53 


4.32 


0.77 


54. 


Financial aid information 


5.00 


0.00 


4.54 


0.69 


55. 


Financial aid contact 


4.63 


0.52 


4.25 


0.82 
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Table 3 Continued 



Survey Item National Panel State Panel 
Mean SD Mean SD 



56. 


Programs offered 


5.00 


0.00 


4.65 


0. 


63 


57. 


Certificates or degrees awarded 


5.00 


O.GO 


4.54 


0. 


,63 


58. 


Length of time to complete 


4.75 


0.46 


4.47 


0. 


60 


59. 


Special programs 


4.13 


0.64 


4.28 


0. 


74 


60. 


Library and learning center 


3.88 


0.64 


3.87 


0. 


75 


61. 


Support services 


4.00 


0.76 


4.08 


0. 


80 


62. 


Services for handicapped 


4.50 


0.53 


4.40 


0. 


69 


63. 


Intercollegiate sports 


3.63 


0.74 


3.76 


0. 


85 


64. 


Accreditation 


4.13 


0.64 


4.21 


0. 


84 
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Table 4 

INFORMATION VALIDATION SURVEY: FINANCIAL AID & SCHOLARSHIP INFORMATION 
RESPONSES OF THE NATIONAL AND STATE PANELS 



Survey Item National Panel State Panel 







Mean 


SD 


Mean 


SD 


65. 


Name 


5.00 


0.00 


4.69 


0.70 


66. 


Eligibility requirements 


5.00 


0,00 


4.73 


0.67 


67. 


Application procedures and deadline 


5.00 


0.00 


4.65 


0.72 


68. 


Size or basis o£ award 


4.86 


0.35 


4.57 


0.73 


69. 


Contact person 


4.50 


0.76 


4.44 


0.80 


70. 


How to obtain add'l information 


4.50 


0.76 


4.57 


0.58 
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Table 5 

INFORMATION VALIDATION SURVEY: SYSTEM DESIGN CONSIDERATIONS 
RESPONSES OF THE NATIONAL AND STATE PANELS 



Survey Item National Panel State Panel 







Mean 


SD 








71. 


Interactive search of occupations 


4.63 


0.52 


4.58 


n 


ft s 


72. 


Ability to query why not 


4.50 


0.53 




n 

u 


. / o 


73. 


Direct access to occupational info 


4,88 


0.35 


4 78 


n 




74. 


Direct access to secondary ed info 


4.50 


0.53 


4 'il 






75. 


Direct access to post-sec prog info 


5.00 


0.00 


4 fi1 






76. 


Interactive search of ed inst'ns 


4.50 


0.76 


4.40. 


0. 


.74 


77. 


Direct access to ed inst'ns info 


4.88 


0,35 


4.42 


0. 


.62 


78. 


Can match progs and ed inst'ns 


4.88 


0.35 


4.64 


0. 


,68 


79. 


Links to assessment instruments 


3.88 


0.83 


3.92 


0. 


84 


80. 


Can sort for occs by other occs 


4.75 


0.46 


4.34 


0. 


78 


81. 


Interactive search of financial aid 


4.63 


0.52 


4.35 


0. 


78 


82. 


Can operate on microcomputer 


4.63 


0.74 


4.66 


0. 


58 


83. 


Provides printed record of session 


5.00 


0.00 


4.81 


0. 


52 


84. 


Can add state specific info 


4.75 


0.71 


4.54 


0. 


58 


85. 


Can add new records for occs & progs 


4.63 


0.52 


4.64 


0. 


51 
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The Surveys of User Satisfaction 

Four types of Surveys of User Satisfaction were sent to each 
of the 41 schools selected. Some responses were received from 
26, or 63%, of the surveyed schools. These returns, however, did 
not adequately represent all of the systems in use in the State. 
The breakdown by CIDS is as follows: 

-Of the 15 CIS sites surveyed, 14 returned responses. 

-Of the 15 GIS sites surveyed, 6 returned responses. 

-Of the 5 CHOICES sites surveyed, 3 returned responses. 

-Of the 4 Discover sites surveyed, 4 returned responses, but 

3 of these indicated they were using GIS. It was not clear 

whether this was instead of or in conjunction with Discover. 

These returns could not be used for system comparisons, but 

they could be used to measure overall responses to CIDS. 

These are referred to henceforth as "mixed sites." 

-No returns were received from either the C-LECT or SIGI 

site surveyed. 

Each site was sent 1 principal's survey, 1 counselor's 
survey, 10 students' surveys and 10 parents' surveys. The 
particular returns by survey instriunent varied within the sites 
responding. The number of returns will be given in the 
discussion of each survey. 

In addition to limitations stemming from the unequal 
distribution of returns, two additional limitations to the 
analyses of the Surveys of User Satisfaction must be noted. 
First, the identification of the sites which were sent the 
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surveys was different for each CIDS. Details of this are in the 
previous chapter. Second, the sites themselves varied greatly 
from large, inner-city schools to small rural schools. There was 
no way to control for this variation nor to measure its impact on 
the results 

The results of the analyses and discussions of their 
implications for the study follow in sections for each survey. 
PRINCIPAL'S SURVEY OF USER SATISFACTION 

Returns were received from 26 principals. Of these 14 were 
from sites using CIS, 6 from sites using GIS, 2 from sites using 
Choices, 1 from a site using Discover and 3 from mixed sites. 

The survey was divided into two parts. In the first part, 
principals were asked whether or not the CIDS had been used in 
their schools to support particular activities. The percentage 
of principals responding positively to each item for the total 
group, and for CIS schools and GIS schools respectively is given 
in Table 6, Principals' User Satisfaction: Use of Cids. 

Items in all the following tables may be abbreviated. The 
actual instruments may be seen in Appendix II. The tables 
related to the Surveys of User Satisfaction follow the discussion 
of these surveys. 

The second part of the survey asked the principals to 
express their degree of agreement on a scale of 5 to 1 with 10 
statements that directly related to satisfaction. With the 
exception of one item, the mean of each of the item scores, for 
all principals was between 4 "agree" and 5 "strongly agree." The 
lowest rated item was, "The system has been used successfully for 
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instruction," 

Mean scores for the statements are presented in Table 7, 
Principals' User Satisfaction: Item Scores. The total group 
score is given, followed by scores for principals of CIS schools 
and then GIS schools. The CIS and GIS scores were compared using 
t-tests. Results significant at the .05 or less level are 
starred. Three items had significant differences. They dealt 
with the general level of satisfaction with the system, the wo • th 
of the system in terms of cost, and the ability to schedule 
students. In all three cases, the principals of CIS schools gave 
higher ratings than the principals of GIS schools. 
COUNSELOR'S SURVEY OF USER SATISFACTION 

The Counselor's Survey of User Satisfaction was returned by 
24 counselors: 12 from CIS schools, 6 from GIS schools, 3 from 
CHOICES sd^ools, 1 from a Discover school and 2 from mixed 
sites. 

The first 10 questions on the survey asked the counselors 
whether or not they had used the CIDS to support particular 
activities in their school. Til'is was essentially the same 
question as that asked the principals. The next set of questions 
asked the counselors to indicate which of 10 student outcomes 
they had observed as a result of students using the CIDS 
individually or in groups. 

Table 8, Counselors' User Satisfaction: Use of CIDS and 
Outcomes shows the percentage of counselors responding positively 
to the items for the total group, for CIS schools and for GIS 
schools. In general, it may be noted that a greater percentage 
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of the counselors from schools using CIS responded positively Co 
the student outcome \tems than did counselors from schools using 
GIS. 

The next set of 20 questions in the survey of counselors 
listed information components and system design considerations 
drawn from the" review of the literature and the Information 
Validation Survey. For each item, counselors were asked the 
"degree of usefulness of each item as it is presented in CIDS you 
are using in your school." The possible answers were: 

5 « Extremely useful 

4 » Very useful 

3 = Moderately useful 

2 " Not very useful 

1 - Not at all useful 

0 " Not applicable. 

Mean responses were computed for all counselors and 
separately for counselors from CIS and GIS schools. T-tests were 
used to examine the differences in the means between the latter 
cwo groups. Two items were found to have significant 
differences: "Financial aid and scholarship information," and 
"Employability skills information." In both cases, CIS was 
scored higher than GIS. It may also be observed from the 
analysis presented in Table 9, Counselors' User Satisfaction: 
System Components, that the components were consistently rated 
higher by the counselors from schools using CIS than by the 
counselors from schools using GIS or by the total group of 
counselors responding. 
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Counselors also responded to questions about training and 
user materials with 79% saying they had received training and 777. 
saying they had a counselor's handbook or implementation guide. 
Those who had training rated it 3.61 on the same scale of 
usefulness, and those who had handbooks rated their usefulness at 
3.27. 

Counselors estimated that the average time session per user 
was 36 minutes and gave their ideal length of time for a 
session as 34 minutes. Overall, 75% of the counselors rated 
their CIDS as easy for students to use, and 75% also rated the 
information that students receive as easy to understand. 

In general, counselors seem only moderately pleased with the 
CIDS used by their schools. None of the mean ratings for 
usefulness of system components, training, or materials reached 
4.0. While 14 of these items were rated between "moderately 
useful" and "very useful," 6 were rated between "not very useful" 
and "moderately useful," and 2 were rated between "not at all 
useful" and "not very useful." 

Counselors' comments in the open-ended question reflected 
some of their ideas about system improvements. Among the 
suggestions made by counselors were the following: 

-More frequent updating of the system; 

-Average salary in New York State for specific careers; 
-Lists of companies that hire; 

-A terminal with printout capabilities for all counselors; 
-Greater interactivity; 
•-Employment opportunities; 
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-Interest inventories and surveys; and 
-Greater availability for more students. 
STUDENT'S SURVEY OF USER SATISFACTION 

The Student's Survey of User Satisfaction produced 179 
responses from 26 schools. Of these 111 had used CIS, 24 had 
used CHOICES, 23 had used CIS, 5 had used Discover, and 16 were 
from the aiixed sites. The mean grade level reported by the 
students was 11, with 16 freshmen, 21 sophomores, 80 juniors, and 
52 seniors responding to that question. Students' responses to 
the question regarding how often they had used the system ranged 
from once or twice through many times to many times per week. 

Students weve asked to respond "yes" or "no" to a series of 
questions regarding the ease of use of the system and the 
helpfulness of the information they found. In Table 10, 
Students' Levels of User Satisfaction, the percentage responding 
positively to each item is given for the total group, for those 
using CIS, those using CIS and those using CHOICES. In general, 
students liked using the CIDS and found them easy to use. More 
students found the CIDS useful in gaining information than in 
making career decisions. This would seem to support the role of 
CIDS as a tool or resource to be used in conjunction with 
counseling and career education activities. 

An atialysis of variance, using dummy coding, was carried out 
for the percentage responding positively by CIS, CIS and CHOICES 
users. Three items reached the required level of s?ignif icance. 
Comments on the limitations of the population surveyed made in 
the introduction to this chapter should be noted. 
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An analysis of variance by grades was also carried out for 
each of the items in the students' survey. Only question 6 
achieved a level of significance of p<.05. In response to the 
item, "Did the computerized systen help you learn more than you 
knew before about how to make a decision about a career?" 93.8% 
of the ninth graders, 81.0% of the tenth graders, 58.87. of the 
eleventh graders and 68.67. of the twelfth graders responded 
"yes." 

PARENT'S SURVEY OF USER SATISFACTION 

The Parent's Survey of User Satisfaction d rew 123 responses. 
Of these, 64 were from CIS schools, 31 from GIS schools, 18 from 
CHOICES schools, 3 from a Discover school and 7 from mixed sites. 

The survey had two distinct sections » The first section of 
five questions was designed to assess the level of satisfaction 
with career information in their children's schools. The second 
section was designed to elicit parents' preferences for the 
content of computerized CIDS. 

The first question asked parents if they thought their 
children's high school provided "the information needed for 
making career ind educational choices." Of all the parents 
responding, 91.07. said "yes." Analysis of variance, however, 
showed a significant ditierence among CIS, GIS and CHOICES 
schools. The percent responding positively for CIS was 93.7; for 
GIS, 93.67.; and for CHOICES 72.27.. 

The next question, whioC also achieved significance in a 
comparison of responses grouped by the CIDS of the children's 
schools, asked parents whether their children had ever brought 
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home a printout from a computerized system at school. The 
percent responding positively for CIS was 92.1%, for CIS was 
96.8%, and for CHOICES was 56.3%. It may be that the 
satisfaction expressed by parents with career information is 
related to the level of communication practiced by the school, 
rather than the system in use. 

Of the parents who had seen a printout, 81.0% though it 
provided valuable information. Only 28.6% of the parents had 
ever seen their child use a computer in school to get 
occupational or college information. Of these, 100.0% thought 
the computer was a valuable tool. For these questions, there 
were no significant differences among the systens. 

The second part of the parents' survey consisted of 10 
items. Parents were asked to respond "yes" if they thought a 
computer would be a good tool for the purpose listed, or "no" if 
thev thought a computer would not be a good tool for that 
purpose. The analysis, presented in Table 11, Parents' 
Preferences for CIDS' Contents, shows that parents generally 
supported the use of computers to deliver career and educational 
information, with one strong exception. More than 50% of the 
parents said they did not see the computer as a good tool to 
"clarify a student's values." 
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Table 6 





PRINCIPALS' USER SATISFACTION 


: USE OF 


CTD*5 






Activity 


7o Total 






C 






N«26 




li — o 




1 • 


Indlviaual career counseling 


96.2 


100 0 


oo 


0 
• O 


2. 


Career education groups or classes 


84.6 


100 0 


1 on 


n 


3. 


College guidance 


100.0 


85.7 


83 


.3 


4. 


Special education programs 


76.9 


85.7 


66 


.7 


5. 


Occupational education prograais 


80.8 


85.7 


66, 


.7 


6. 


Basic skills or remedial reading 


23.1 


85.7 


66, 


.7 


7. 


Subject class instruction 


53.9 


64.3 


50. 


.0 


8. 


Job placement 


30.8 


21.4 


50. 


,0 


9. 


Employability skills classes 


42.3 


35.7 


66. 


7 


IC. 


Counseling physically handicapped 


36.0 


38.5 


50. 


0 
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Table 7 

PRINCIPALS' USER SATISFACTION: 



ITEM SCORES 



Item 



11. Satisfied in general* 

12. Students have said CIDS useful 

13. Information is appropriate 

14. Parents have said satisfied 

15. System used for instruction 

16. System accepted by counselors 

17. System worth the cost* 

18. CIDS helps counselors use time 

19. Training time appropriate 

20. No problem scheduling CIDS use* 
*«CIS V. GIS, p<.05. 



Total 
Mean (SD) 
N=26 



CIS 
Mean (SD) 
N=14 



GIS 
Mean (SD) 
N=6 



4.58( .58) 

4.46(.65) 

4.54( .51) 

4.04(.79) 

3.42(1.04) 

4.50(.71) 

4.62( .50) 

4.19(,80) 

4.44(.65) 

4.19(1.02) 



4.79( .43) 

4.57(.51) 

4.64( .50) 

4.00(.91) 

3.50(1.09) 

4.43(.51) 

4.79( .43) 

4.29(.73) 

4.54( .52) 

4.57(.65) 



4.17( .75) 

4.33(.82) 

4.33(.52) 

4.00(.89) 

3.00(.63) 

4.50(1.23) 

4.33(.52) 

4.33(.82) 

4.50(.55) 

3.67(1.37) 
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Table 8 

COUNSELORS' USER SATISFACTION: USE OF CIDS AND OUTCOMES 



Activity 


% Total 


%CIS 


%GIS 






N=24 


N=12 


N=6 


1. 


Individual career counseling 


75.0 


91.7 


100.0 


2. 


Career education groups or classes 


66.7 


83.3 


83, 3 


3. 


College guidance 


79.2 


100.0 


100.0 


4. 


Special education programs 


43.5 


58.3 


60.0 


5. 


Occupational education programs 


65.2 


83.3 


80.0 


6. 


Basic skills or remedial reading 


21.7 


. 33.3 


20.0 


7. 


Subject class instruction 


43.5 


58.3 


60.0 


8. 


Job placement 


39.1 


58.3 


40.0 


9. 


Employability skills classes 


31.8 


43.5 


40.0 


10. 


Counseling physically handicapped 


52,2 


58.3 


80.0 


11. 


Increased awareness of self 


66.7 


91.7 


66.7 


12. 


Knowledge of skills transferability 


66. 7 


91.7 


66. 7 


13. 


General knowledge of labor market 


75.0 


100.0 


83.3 


14. 


General knowledge about careers 


75.0 


100.0 


83.3 


15. 


General info about jobseeking skills 


50.0 


83.3 


16.7 


16. 


General info about education options 


75.0 


iOO.O 


83,3 


17. 


Specific ed Institutions info 


79.2 


100.0 


100.0 


18. 


Gained info about financial aid 


70.8 


100.0 


83.3 


19. 


Made a decision about career choice 


69.6 


90.9 


83.3 


20. 


Made a decision about further educ. 


70.0 


100.0 


83.3 
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Table 9 

COUNSELORS' USER SATISFACTION: SYSTEM COMPONENTS 



Total CIS CIS 







Mean (SD) 


Mean 


(SD) 


Mean (SD) 






XT O /. 


N=12 




N=6 


21 


v/v»uu[^aL. J.v^lla JL UCSCnptLOnS 


3.54(1.98) 


4. 58( 


.52) 


4.17(1.17) 


22. 


V/ wv* U|./a L. J. uila J. CJLuISl6iS 


i. 08(1. 74) 


4.00{ 


.60) 


3.67(1.03) 


23 r 


Rninl ovmpnt* oiit*l <^<^lr ■! n^i^vmof**? 


i. 08(1. 86) 


3. 92( 


.79) 


3.67(1.51) 


24. 


LJoOO ay^f^ col 01*\r 4 r^^y^w■l«ftf••^ 


2.88(1.68) 


3.67( 


.78) 


3.33(.82) 


25. 


Phvfij_r*Al HpmAnHQ onri or> t*-f \r-f t* 4 oe 


o f\/, / 1 "yc \ 


3. 83( . 


.72) 


3.67(1.03) 


26. 


Licensing and certification 


2.83(1.76) 


3.92( , 


.79) 


3.00(1.41) 


27. 


Information about related occ'ns 


3.04(1.83) 


4.08( , 


.52) 


3.33(1.63) 


28. 


Info about ed progs related to occs 


3.42(1.93) 


4.42( . 


.52) 


4.00(1.27) 


29. 


Descriptions of educational progs 


3.38(1.91) 


4. 50( , 


,52) 


4.00(1.10) 


30. 


Info about occs related to ed progs 


3.13(1.92) 


4.00( . 


74) 


3.67(1.75) 


31. 


Educational institutions descrip'ns 


3.17(1.90) 


4. 17 ( . 


58) 


3.67(1.75) 


32. 


Financial aid/scholarship info* 


2.67(1.81) 


3.92(. 


79) 


2.83(1.17) 


33. 


Employability skills information* 


1.58(1.72) 


2.67(1 


.67) 


.50(.55) 


34. 


User (student) handbooks 


3.21(1.96) 


4.33(. 


65) 


3.33(1.63) 


35. 


Interactive occupations exploration 2.43(1.78) 


3.09(1 


.30) 


2.83(1.47) 


36. 


Ability to ask why not 


2.96(1.92) 


4.00(. 


74) 


3.00(1.90) 


37. 


Interactive ed. inst'ns exploration 2.30(1.82) 


3.12(1 


.33) 


2.17(1.60) 


38. 


Method of direct access to occs 


3.30(2.10) 


4.27(. 


79) 


4.00(2.00) 


39. 


Methods of direct access to ed info 


3.17(2.12) 


4.27(. 


65) 


3.50(2.35) 


40. 


Links to assessment instruments 


1.50(1.98) 


1.75(1 


.91) 


1.67(2.25) 



*-CIS V. CIS, p<.05. 
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Table 10 

STUDENTS' LEVELS OF USER SATISFACTION 



Question 


7, Total 


%CIS 










N-179 


N=lll 






1. 


Were you given a user handbook?* 


88.8 


99.1 


73.9 


83 3 


2. 


If yes, was it useful? 


98.7 


100.0 


100 .0 


100 1 


3. 


Was the system easy to use? 


95.5 


93.6 


100 0 


1 00 n 


4. 


Information easy to read/ understand? 


97.2 


99.1 


95. S 


1 00 n 


5. 


Learn something new about occs?* 


85.9 


89. 1 


72 7 


7 J • o 


6. 


Learn how to make a career decision?*68.0 


70.3 


50 0 


87 S 


7. 


Understand occs & your interests?* 


79.2 


78.4 


68. 2 

\J yy % 4m 


Q S R 
7 J • o 


8. 


Understand occs & your abilities? 


74.0 


74.6 


11 .1 


91.7 


9. 


Relationship occs & education progs? 


87.6 


87.3 


86.4 


95.8. 


10. 


How to prepare for an occupation? 


75.3 


80.2 


81.8 


79.2 


11. 


Help you make a career decision? 


55.6 


55.0 


59.1 


70.8 


12. 


Did you get a printout? 


94.4 


98.2 


95.5 


91.7 


13. 


Was having a printout useful? 


93.7 


96.4 


95.5 


87.5 


14. 


Did you like using a CIDS? 


97.2 


98.2 


100.0 


95.8 


15. 


Recommend the CIDS to someone else? 


98.9 


100.0 


100.0 


100.0 


* m 


CIS V. GIS V. CHOICES, ANOVA P<.05. 
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Table 11 

PARENTS' PREFERENCES FOR CIDS' CONTENTS 

~~ ' %Total 
— ■ , N==123 

4. Identify careers based on student's interests and abilities 95.1 

5. Identify colleges which meet student's needs and interests 98.4 

6. Clarify a student's values 47.1 

7. Show relationship between work and educational programs 86.1 

8. Deliver information about occupations 98.4 

9. Deliver information about areas of concentration in h.s. 84.4 

10. Deliver information about educational programs in college 98.4 

11. Deliver information about colleges and universities 99.2 

12. Deliver information about private trade schools 92.9 

13. Deliver information about financial aid and scholarships 93.5 
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The Site Visits 

In addition to surveying schools in the State which were 
using CIDS, visits were made to sites in which the various 
systems were being utilized. The purpose of the visits was to 
examine the systems in schools and agencies to query counselors 
about their opinions of the system's on an informal basis. The 
questions which were asked were designed to obtain information 
which would not emerge on the surveys. 

In an effort to provide some structure to the visits, a 
questionnaire was designed. A copy is attached in Appendix IV. 
The four questions were: 

1. What are the major programmatic goals of the career 
counseling program in your school (or agency)? 

2. What are the most important kinds of information needed 
by the clients you serve to make career and educational 
decisions? 

3. How can a computer-based CIDS help meet these goals and 
deliver information? 

4. What are the major strengths and weaknesses of the system 
you use (include training, ancillary materials, length of 
sessions, etc.)? 

In addition to the answers to the questions, the name and 
address of the school or agency, the date of the visit, the name 
and title of the respondent or respondents, the number of clients 
served, and the length of time the system had faeen in use were 
recorded. At the end of the session, the counselors were asked 
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if they wished to discuss issues or questions which had not been 
asked. 

Visits were made to five high schools, one local vocational 
rehabilitation office, and one junior college. One of the high 
schools was using Discover ru.:-.ing on both Apple and IBM 
microcomputers. They were completing their second year with 
Discover, after changing from GIS during the previous year. One 
school was using CHOICES with five microcomputers. The school 
which was using GIS had a computer terminal and was accessing the 
on-line system from a time-share computer. Another school was 
using MetroGiiide with a microcomputer and modem, with some stand 
alone software. The school which was using two systems had GIS 
on-line from a main frame computer and Discover on a micro- 
computer. The vocational rehabilitation office was using 
CHOICES, and the junior college was using SIGI, both with a 
microcomputer. 

The schools included two which had students in grades ten 
through twelve, and three which served students in grades nine 
through twelve. The titles of those who were interviewed were 
director of guidance, assistant principal for /guidance, guidance 
chairman, and two guidance counselors. The length of time they 
had been using the CIDS ranged from one year to 15 years. 

The smallest school had approximately 800 students. The 
other four had student enrollments of between 1050 and 1900. 
The number of students who had used at least one component of the 
school's CIDS during the past year ranged from 150 to 1050. 

At the vocational rehabilitation office, the vocational 
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rehabilitation counselor was interviewed. The system there had 
been in use for almost three years. It was his estimate that 
between 140 and 150 clients were served by the agency's 
computerized career information delivery system out of the total 
of 3000 a year who received some service from the office. The 
age of the clients who used the system during the past year 
ranged from 18 to 65. 

At the junior college the director of career services was 
interviewed. The clients which the institution serves are 18 to 
19 years old. He described them as "commuter, non-traditional" 
students. Approximately 300 of the 1000 to 1200 students 
enrolled have used the system each of the five years it has been 
in operation. 

The responses from the sites visited are presented in the 
four sections below. Those for the first two questions are 
presented by type of institution visited, since both goals and 
the level of information needed tended to vary according to .type 
of client served. The responses Co the third and fourth 
questions are combined. In general, there was consensus on the 
uses of a computerized system to help meet goals. The 
conclusions drawn from the discussion of the strengths and 
weaknesses of systems are presented as a general overview of what 
counselors say they think are important components of a CIDS. 
CAREER COUNSELING GOALS 

Generally in the schools the goals of the counseling program 
were defined in terms of grade levels. Overall, the ninth grade 
goals were to develop career awareness and define work and 



o 139 

ERIC 



CIDS Evaluation, 127 



personal values. The goal for vocational students was to help 
them find a program which matched their interests and personal 
needs. At the tenth grade level, the goals were directed toward 
helping the students find out more about occupational choices. 
The activities included career exploration, career decision 
making, and developing decision making skills. 

For the eleventh grade, the goals were most often described 
in terms of assisting in the exploration of occupational and 
educational options and in clarifying post-secondary plans. 
Activities such as college search (including applying to college 
and getting financial aid), determining possible college majors, 
and developing job seeking and getting skills were most often 
mentioned. By the twelfth grade, the goals were primarily to 
assist with college choice, college entrance procedures, college 
and financial aid applications, resume writing, and filling out 
job applications. 

The overall goal of the vocational rehabilitation program 
was to place clients in jobs. The counseling program supports 
this overall goal by assisting clients in establishing vocational 
goals. Since the psychological workups provided to clients do 
not include job specific recommendations, the CIDS is used to 
help sharpen the focus of clients in setting their vocational 
goals. The use of the system was described as assisting the 
agency to help their clients by diminishing the chances of 
misfits between people and jobs. 

At the junior college, the counseling program goal was 
described primarily as to assist students with self-assessment, 
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values clarification, and vocational exploration. 
INFORMATION NEEDS OF CLIENTS 

One school stated that since they have a pro-active approach 
to counseling, they encourage broad exploration and self 
assessment by their students. Self evaluation is dynamic and 
encourages adaptability on the part of the student. For this 
program, the CIDS structured search with variables, such as 
interests, aptitudes, and temperaments is a valuable tool. 

The occupational information components most often mentioned 
by the school counselors were: 

job descriptions (must be well done); 

job duties; 

interests related to a job; 

aptitudes; 

temperaments; 

education and training required for a job; 
working conditions; 
employment outlook; and 
earnings. 

The educational information described as most useful 
included: 

location and size of the school, size of the city; 
majors/program of study information (important it be 
accurate) ; 

campus activities available: sports, social; 
financial aid available: type, number of students receiving 
aid, average award, application procedures; 
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entrance requirements: scores, applications, deadlines; 
costs; and 

good overall profiles of colleges. 

The information cited as needed by clients at the vocational 
rehabilitation office included primarily what people who are 
changing occupations need to know to help insure that the change 
is not random. The system and information components included: 

interest identification; 

search routine; 

physical demands of a job; 

occupational description; and 

job seeking skills. 

The director of career services at the junior college listed 
the following system functions and information components as 
being important: 

computerized self-assessment routine; 

routine to identify and prioritize interests and values; 
vocational exploration; 

education and training required for a job; 
occupational description; and 
methods of entry into an occupation. 

It is interesting to note that the information mentioned by 
those interviewed during the site visits agrees with information 
components and system design features which received the highest 
ratings on the Information Validation Surveys and the Surveys of 
User Satisfaction. 
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CIDS USES 

When asked how a CIDS could help the school or agency to 
meet counseling program goals and deliver information, there was 
general agreement that computers are a good way to deliver career 
information and provide an excellent opportunity for clients to 
find the information they need on an individual basis. It should 
be noted that the examples given by counselors at all three types 
of sites visited were very similar, and that a few statements 
were made by all interviewed. 

The advantages of using a computerized system which were 
most frequently discussed included the fact that computers are 
non- judgmental, and the users feel chey get objective answers 
from the computer. Further, clients get printouts of the 
information obtained from "the session at the computer to take 
home, and this allows their parents or other family members to 
become involved in their career choices. It was pointed out by 
several counselors that their CIDS is cost effective in that it 
provides an inexpensive source of large amounts of information. 

Since users get immediate feedback from the computer during 
a session, they have the opportunity to see the results of their 
choices, to change choices, and then see the results of their 
changes. This broadens their horizons and encourages them to 
engage in "what if" strategies without risk. It is an effective 
way of helping people develop decision making skills. 

Most clients begin their career exploration and decision 
making processes without any idea of what career they want or of 
what might be available to them. One goal of career counseling 



143 

ERIC 



CIDS Evaluation, 131 



programs is t:o correct this situation. A computerized system 
allows users to delve into the labor market over and over from 
different directions, to test themselves against the conditions 
of occupations or occupational groups, and thereby help increase 
the number and types of opportunities which might be available to 
them. 

The one area which was mentioned only by a school was the 
usefulness of their CIDS in teaching the new occupational 
educational programs, and particularly "Home and Career Skills". 
They found the computerized system to be extremely beneficial to 
both students and teachers in this activity. 

In general, there was agreement that computerized systems 
are an excellent counselor tool, because vast amounts of 
information can be stored and made readily available to users. 
One counselor pointed out that the human brain cannot recall 
quickly minute details about occupational and educational 
possibilities, as can a computer. However, counselors stressed 
the caution that it should be remembered that a CIDS is only a 
tool. It cannot take the place of the counselor; personal 
intervention by the counselor is essential. Further, CIDS are 
most useful when they are used with other sources of information 
and not offered as the only counseling tool available. 
STRENGTHS AND WEAKNESSES OF CIDS 

While the individual responses to this question were 
directed toward the specific systems the respondents used, the 
strengths and weaknesses mentioned are combined without 
mentioning the name of the CIDS. Obviously, the opinion of one 
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or two people about the strengths and weaknesses of any system 
cannot be considered a valid judgment of that system. However, 
the responses taken together do give a good picture of what 
counselors want and need in a computerized career information 
delivery system to help them serve their clients' needs and to 
meet their program goals. 

Whether cited as a strength or weakness, the system and 
information components which emerged as being important to 
counselors included: 

-The client and counselor manuals should be complete and 

easy to use. 

-Training should be adequate to prepare counselors to make 
maximum use of the CIDS* 

-The CIDS should be easy for the clients to use; should be 
user friendly. 

-The computer printouts should be readable, easy to 
understand, and clear as to what information is being given. 
-The language contained in the CIDS should be at a level 
which clients can understand. 

-The CIDS should be designed to be very flexible, with 
several routines and ways to get at information. 
-It should contain good links to available printed resources. 
-The search routine in CIDS should be well designed so the 
results make sense to the users. 

-The information about occupations must be accurate, 
current, interesting, and useful. 

-The information about colleges should be up-to-date, 
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complete, and accurate. 

-Information about financial aid is important. 

-The length of and average sessions should fit into a class 

period. 

In general, the findings of the site visits showed the 
following. As would be expected, the goals of c.reer counseling 
programs vary depending on the type of institution and the 
clients served, and the specific types of information needed to 
meet those goals varies by level of detail and type. However, 
there is much similarity between agencies in the ways CIDS can 
be used and are being used to meet these goals. In addition, 
there is great similarity between counselors from different 
agencies in what they feel are the important components of a 
computerized system, training, and system support materials. 

In summary, the information obtained from the site visits 
confirmed and reinforced the findings concerning information 
components and system design considerations obtained from the 
surveys. 
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The Information from Vendors 

The responses to the request for information (RFI) and the 
ratings of the System Review Panel were combined to achieve a 
full picture of the CIDS under review in this study. Six vendors 
responded to the RFI. However, only five vendors completed the 
presentation and demonstration to the System Review Panel. 
Therefore, the analysis of the RFI response was limited to the 
five vendors completing the entire process. It should be noted 
that the initial letters of invitation sent to the vendors 
stipulated the requirements for participation. The responses 
analyzed came from C-LECT, CHOICES, CIS, Discover, and CIS. (In 
all analyses, the vendors will be presented in this alphabetical 
order.) 

The RFI sought information on system components and 
features, machines and media available, related research and 
evaluation studies, and costs about each system from the vendors. 
In addition, all vendors were asked to supply current printouts 
from their CIDS on a partiqular occupation and institution. Each 
of the areas of the RFI will be discussed in the sub-sections 
which follow. 

INFORMATION COMPONENTS, SYSTEM DESIGN, AND SUPPORT MATERIALS 
Information Components 

The vendors were asked to check a list of career information 
items contained their CIDS, compiled on the basis of the results 
of the Information Validation Survey and the Surveys of User 
Satisfaction. For each item, the vendor was asked to check all 
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that applied: 

-NATIONAL, indicates that your system currently carries 
information on this item representing a national level of 
data analysis; 

-NYS, indicates that your system currently carries 
information specific to New York State for this item; 
-ABLE, indicates that yov:r system is currently able to carry 
state-specific information for this item if it were supplied 
to you. 

As described in the previous chapter, "Methodology," and in 
Appendix VII, Instruments for Evaluating CIDS; items could be 
counted for one or all three of the levels of analysis and could 
carry a weight of one to six points. 

The points for the vendors' statements of their CIDS' 
information components were counted and expressed as a percentage 
of the possible total. The scores for information components, 
thus achieved, were compared on three bases: national information. 
New York State information, and ability to add state information. 

In Table 12, Information from Vendors: National 
Information, it can be seen that the range of scores was from 
72.27% to 91.59%, with CIS at the highest, C-LECT following, and 
the remaining three grouped together at 72%. This table and the 
others related to the RFI are found after this discussion of the 
RFI findings. 

Table 13, Information from Vendors: New York State 
Information, presents a similar analysis of New York State data 
which was reported as available. The discrepancies between 



148 



CIDS Evaluation, 136 



systems was much greater. CIS scored 95.387.. The next closest 
was C-LECT with 58.46X, and the lowest on this factor was CHOICES 
with 13.85Z. It may be noted, at this point, that CIS operates 
only as a state- or local-based system, and is operating in New 
York State, as MetroGuide, the New York City system. 

The ability of the systems to add state information is 
presented in Table 14, Information from Vendors: Ability to Add 
State Information. CIS and CHOICES earned over 90 percent, with 
CIS at a high of 98.45%. The lowest score on this was Discover 's 
68.38%. Discover commented in their response that they were 
limited by file space in their ability to add information. 
System Design Considerations 

The fifteen items of system design considerations were 
analyzed similarly using a weighted scale. Table 15, 
Information from Vendors: System Design Considerations, shows 
that all systems had most of the features desired by respondents 
to the Information Validation and User Satisfaction Surveys. 
System Support Materials and Training 

The six weighted items describing vendors' system support 
materials and training were analyzed in the same way. Table 16, 
Information from Vendors: User Materials and Support, shows that 
four of the systems ~ C-LECT, CHOICES, CIS, and CIS -- reported 
supplying 100.00% of the materials and services. 
Score on Information, System Design, and Support Materials 

Table 17, Information from Vendors: Total, All Information, 
combines the scores earned on all the system and ancillary 
features discussed above to give a picture of the overall content 
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and capability of each of the systems being evaluated. 

The highest overall score was earned by CIS, 94.00%; second was 

C-LECT at 79.00%; third, GIS, 68.00%; fourth, CHOICES 67.34%; and 

fifth. Discover, 62.00. 

LENGTH OF SESSION 

Related to the contents and features of a system is the 
length of time it takes students or clients to complete a session 
in which they get information meaningful to them. Obviously, 
this can vary greatly depending upon the age, stage, and 
capability of the user. To get similar figures, the vendors were 
asked. 

Approximately how much time does it generally take for 

average high school students to complete a session in which 

they use the occupational exploration strategy or strategies 

and generate a list of occupations meaningful to them. 

Counselors responded in their survey that that the ideal 
length of time for a session was 34 minutes. In the site visits, 
counselors expressed this as "less than a class period." The 
figures supplied by the vendors are: 

C-LECT: 40 minutes; 

CHOICES: 15 to 45 minutes; 

CIS: less than one class period; 

Discover: 5 to 60 minutes; and 

GIS: 15 to 30 minutes. 
MACHINES AND MEDIA 

The vendors were asked to identify all the time-shared and 
microcomputer versions of their systems currently available and 
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supported by their organization. No attempt was made to evaluate 
this information, since it is not the number of computer versions 
available that is important, but whether the software runs on a 
particular computer available to an interested school or agency. 
The list of computers identified by any of the systems and all of 
the systems running on each of those computers is given in Table 
18, Hardware Versions: Mainframe Computers and Microcomputers. 

In addition to identifying hardware versions of their 
systems, vendors were asked to describe any delivery media 
currently available and supported by their organizations "which 
would facilitate classroom delivery of occupational exploration 
and occupational information retrieval." 

All of the vendors described a variety of methods for 
utilizing their CIDS in classrooms. One of the ways of using 
CIDS that is becoming increasingly possible is through the use of 
microcomputer labs and/or through the use of a microcomputer with 
a large screen. The importance of the vendors' responses lies in 
their support of the entire idea of using CIDS in relation to the 
new occupational educatvoTi curricula. The following list is 
therefore not exhaustive but presents only some of the 
outstanding features of each system. 

C-LECT: occupational briefs, occupational view deck, 
workbooks ; 

CHOICES: activities checklist, audio-visual orientation to 

the system, on-line "junior" system; 
CIS: implementation handbook with lesson plans, needle sort 

deck, personal user's handbook; 
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Discover: career journey film, short form of Career Planning 
Program; and 

GIS: posters, filmstrips, other printed materials. 
COMPREHENSIVENESS OF INFORMATION 

One of the concerns expr2ssed by respondents of the surveys 
and by guidance personnel interviewed during the site visits Wc.s 
the accuracy and comprehensiveness of the information. In the 
RFI, vendors were requested to provide a printout of "all 
information which users would obtain if they used the direct 
access route to information about the occupation Physical 
Therapist" and "all information which users would obtain if they 
used the direct access route to information about the post- 
secondary institution State University of New York at Albany." 
All vendors provided this information. 

Rating guides were prepared for both the occupational and 
educational institution information. For the rating guides, 
specific information components were identified on the basis of 
their usefulness and the general availability of reliable 
information to analyze the component. Each component was rated 
for comprehensivenss and accuracy on a scale of 5 to 1 , 
"excellent" to "atrocious." Details of the method of analysis 
are found in the chapter, "Methodology," and in Appendix VII, 
Procedures and Instruments for Evaluating CIDS. 

Twelve occupational and eight educational information 
components were analyzed. The ratings for each system on each 
item as well as the total score are shown in Table 19, Ratings of 
CIDS: Occupational and Educational Information. 
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Several observations about these ratings may be made. None 
of the systems had uniformly excellent data. All were missing or 
had minimal information on from six to eight components. 

In general the occupational information was rated higher 
than the educational information. Whether this was a function of 
the quality of the information or of a lack of inter-rater 
reliability cannot be assessed. However, a letter which 
accompanied the ratings provided by the Office of Higher and 
Professional Education drew attention to a number of gaps and 
errors in the printouts, and urged the incorporation of ::he 
information they develop for the benefit of New York State students. 
RESEARCH AND EVALUATION STUDIES 

The vendors were asked to identify and describe briefly any 
studies of their systems which related to the validity of their 
instruments, the validity of their occupational data, and/or the 
use of their systems in secondary school classroom or group 
settings. 

All of the vendors supplied some information in response to 
this question, but it varied greatly in nature. A brief summary 
of the response by the vendors to each of the points above 
follows. 

C-LECT described validation of its instrument. Temperament 
Survey, and described how they validated occupational information 
through reviewers. They had no research on classroom use. 

CHOICES provided a list of 37 studies and reports but no 
summaries nor statements of the relationship to the three points 
in question. In addition, a full copy of a study to assess the 
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impact of CHOICES as an adjunct to career counseling and 
classroom activities was attached. 

CIS described studies of the ralidity and readability of 
QUEST, the interactive search of occupation, and described their 
use of review panels to validate occupational information. They 
attached a summary of studies on the design and development of 
the system which had appeared in the Annual Review of Research in 
Vocational Education. 

CIS cited research on the reliability and validity of the 
Career Decision-Making System, which is sold by CIS as a product 
for use with CIS or by itself. No further research studies were 
cited. 

DISCOVER cited research studies related to the validation of 
its instruments, the validation of its information through 
mailings to trade or professional associations, and its impact on 
counseling. 
COSTS 

Vendors were asked to supply cost figures for three major 
items using a set of assumptions and a scenario described in 
"Methodology." The reason for the scenario was to have 
comparable figures for all vendors. There were three major areas 
in which costs were examined: 

-basic system costs for delivery to user schools or agencies 

over three years; 

-addition of State data to existing occupational information 
files; and 

-creation of an entirely new file of State data. 
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Basic System Costs 

Vendors were asked to provide the total cost — license fee, 
software fees, training, user handbooks, etc. — of delivering 
their systems for each of three years: In the first year to 100 
microcomputer sites; the second year, 200 sites; and the third 
year, 300 sites. It Is Important to consider the costs for at 
least three years, because systems which may Initially cost more 
often include developmental costs which then diminish with the 
use of the system. In Table 20, Basic System Charges for Three 
Years, both the breakdown of charges by individual items and the 
cost per year for each system can be seen. 
Adding State Data to Existing Files 

Comparing costs for the addition of state-developed data to 
existing files is even more difficult than comparing basic costs. 
One way of adding State data is for the State to enter them into 
the CIDS' software. Two of the systems. Discover and GIS, do not 
permit this and will not provide the source code necessary to do 
it. Furthermore, Discover stated that their file space was 
limited to 425 occupations where the scenario had called for 600. 
The costs for State data entry for three years are given in Table 
21, System Charges for All Components for Three Years. Another 
way for the State to add data is to turn the information over to 
the CIDS. Costs for their entering the data are provided in the 
same table. 
Creation of a File 

Vendors were asked to supply their charges for incorporating 
and delivering a hypothetical file of JTPA program providers. 
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There were great differences in the cost of a new file. Cost 
breakdowns which were provided by all but one vendor did not shed 
much light on the reasons for the differences. Again, the 
figures provided by the vendors are displayed in Table 21. 

It should be noted in examining Table 21 that the costs are 
not additive. For example, there would not be both the cost of 
vendor maintained and state maintained data. Second, activities 
which are not permitted by some vendors do not show up as costs, 
not because they are free, but because they cannot be carried 
out . 

COPYRIGHT POLICIES AND STATE MODIFICATIONS 

Each system was asked to state its policies concerning 
copyrights and use of State modifications tc their system. A 
summary of their responses follows: 

C-LECT: provided their general copyright notice, but did 

not respond to the question; 

CHOICES: modifications to the system are protected by joint 
copyright; the State could not distribute such matarials or 
software outside the State; 

CIS: the State holds title to local modifications and may 
distribute them within the State; local modifications may 
not be distributed outside the State without the written 
permission of both the State and CIS; the State may also 
participate in joint developmental projects for national 
enhancements with CIS and its other operating States; CIS 
holds copyrights to national enhancements. 
DISCOVER: modifications to Discover made at the State's 
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request will be copyrighted and owned by ACT 
GIS: State data added to GIS remains the property of the 
State and cannot be released by Houghton Mifflin without 
the permission of the State. 
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Table 12 

INFORMATION FROM VENDORS: NATIONAL INFORMATION 



- d 



:3.7! 




/ / / > 
■■/'/// 



V 




72.79 



\////A 



(////A 

' / / / / / 

V////, 

V///// 

• / / / / A 



72.54 




CHOICES 



as 



DISCOVER 



Q5 



ERIC 



158 



Table 13 

INFORMATION FROM VENDORS: NEW YORK STATE INFORMATION 
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Table 14 

INFORMATION FROM VENDORS: ABILITY TO ADD STATE INFORMATION 




Table 15 

INFORMATION FROM VENDORS: SYSTEM DESIGN CONSIDERATIONS 
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Table 16 

INFORMATION FROM VENDORS: USER MATERIALS AND SUPPORT 
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Table 17 

INFORMATION FROM VENDORS: TOTAL, ALL INFORMATION 
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Table 18 

HARDWARE VERSIONS: MAINFRAME COMPUTERS AND MICROCOMPUTERS 



Computer 


C-LECT 


CHOICES 


CIS 


Discover 


GIS 


Apple II 




X 








Apple II (HD) 








X 




Apple 11+ (HD) 


X 


X 








Ap le lie 


X 


X 








Apple lie (HD) 




X 




X 


X 


Apple III (HD) 










X 


CDC Cyber 






X 






Commodore 64 




X 








DEC 10 






X 




X 


DEC 20 






X 




X 


DEC PDP-11 






X 




X 


DEC VAX 






X 


X 


X 


Digital Rainbow 


X 








X 


Honeywell 6600 






X 




X 


HP 3000 






X 


X 


X 


IBM 3000/4000 










X 


IBM 36 








X 




IBM AT 




X 


X 






IBM AT (enhanced) 








X 




IBM PC 




X 








IBM PC (HD) 




X 




X 




IBM XT 


X 


X 


X 


X 


X 


IBM (CMS or TSO) 






X 
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Table 18 Continued 



Cnmputer C-LECT CHOICES CIS Discover GIS 

NCR X 
Prime 

Tandy 1000 (HD) X X 

Tandy 1200 X X 

Tandy 6000 X 

TI 990 X 

TRS 80/12 X X 

TRS 80/16 X X 

TRS 80/11 X 

Univac 90/80 X 
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Table 19 

RATINGS OF CIDS: OCCUPATIONAL AND EDUCATIONAL INFORMATION 
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13. Admission requirements 


1 


1 


^ 


1 


1 


14. Cost 




1 


1 


i 




15 • Institution tvoe 




4 


3 


4 


3 


16, Housing 


1 
X 


2 


3 


2 


4 


17, Graduate program (In Ed) 


0 


1 


2 


2 


1 


18. Services for handicapped 


0 


0 


3 


2 


4 


19, Intercollegiate sports 




-not rated- 




20. Enrollment 


3 


3 


3 


3 


3 


Total out of 95 


41 


64 


62 


64 


68 


Percent score 


43.16% 


67.37% 


65.26% 


67.37% 


71. 5J 
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Table 20 

BASIC SYSTEM CHARGES FOR THREE YEARS 
C-LECT 

i^em Year One Year Two Year Three 

(No cost breakdown provided) 

Totals $ 60,200 $144,600 $260,400 

CHOICES 

Item Year One Year Two Year Three 

License fee to State 0 0 0 

Site license fees $76,000 $144,000 $212,000 

Other software fees 0 0 0 

Counselor training 6,000 4,000 3,000 

Counselor manuals 0 0 0 

Indices 0 0 0 

User handbooks 14,000 38,250 59,000 
(500/site/year) 



Totals $ 96,000 $186,250 $274,500 



Er|c i67 



CIDS Evaluation, 155 



Table 20 Continued 



ERIC 





CIS 






Item 


Year One 


Ypflr Tun 


I 6 a r li. 


License fee to State 


$ 5,050 


^ 5 050 


$ S OSO 


Software license fees 


56,000 


J U , V/V/V/ 




National information 


35,000 


S 000 




Service center support 


17,500 


9fi 9 SO 


O c AAA 
,000 


Indices 


175 


7 SO 


c o c 


Service center management 17,500 


26,250 


35,000 


User services 


59,000 


OO , JUU 


i i Q AAA 
llo ,000 


Counselor handbooks 




2,000 


3 000 

J , V/V/V/ 


User Handbooks 


7,- 00 


12,500 


15,000 


(500/site/year) 








Inflation @57o 


0 


12,595 


15,128 


Totals 


$199,225 


$264,495 


$317,703 



Discover 

i^ejS Year One Year T-: 'o _ Yfar Three 

License fee to State 0 0 0 

Site license fees $175,000 $350,000 $525,000 

Other software fees 0 0 0 

Counselor training 7,500 7,500 7,500 

Counselor manuals 0 0 0 

Indices 0 0 0 

User handbooks 0 q 



0 



(500/site/year) 



Totals $182,500 $357,500 $532,500 
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Table 20 Continued 

GIS 

Item Year One Year Two Year Three 

License fee to State 0 0 Q 

Site license fees $149,162 $295,034 $442,551 

Other software fees 0 GO 

Counselor training 0 GO 

Counselor manuals 4,500 9,000 13,500 

Indices G GO 

User handbooks 97,500 195,000 292,500 
(500/site/year) 



Totals $251,162 $499,034 $748,551 
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Table 21 

SYSTEM CHARGES FOR ALL COMPONENTS FOR THREE YEARS 



Item 



C-LECT 



Year One 
CHOICES CIS 



Discover GIS 



Basic system 60,200 

Vendor maintained State DB 75,200 

State maintained State DB 72,700 

JTPA file 70,200 



96,000 199,225 182,500 251,162 

5,000 10,000 0 25,485 

1,500 5,000 * * 

10,000 8,000 20,250 12,450 



Item 



Year Two 

C-L£Cr CHOICES CIS 



Discover GIS 



Basic system 144,600 

Vendor maintained State DB 180,400 

State maintained State DB 174,400 

JTFA file 168,400 



186,200 
5,000 
1,500 
5,000 



264,495 
10,000 
0 

5,000 



357,500 499,034 

0 15,105 

* * 

20,000 6,750 



Item 



Year Three 



C-LECT CHOICES CIS 



Discover GIS 



Basic system 260,400 
Vendor maintained State DB 324,600 
State maintained State DB 313,300 
JTPA file 303,000 



274,500 317,703 

5.000 10,000 

1,500 0 

5,000 5,000 



532,500 

0 
* 

30,000 



748,571 

19,245 
* 

8,550 



*Does not provide source code for data entry. 

N.B.: System charges are not additive. See discussion. 
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The Demonstration of CIDS to the System Review Panel 

The culminating activity of the study was the presentation 
and demonstration of the CIDS to -he System Review Panel. 
The panel was composed of 33 participants from appropriate State 
and local agencies. Six vendors were invited to participate 
in the demonstration. They were: C-LECT, CHOICES, CIS, Discover, 
GIS, AND SIGI. 

SIGI withdrew from the process at midday after giving their 
oral presentation, but before giving any demonstrations. They 
were unable to operate their system on the equipment they 
brought. (The generosity of other vendors, particularly C-LECT, 
who offered to lend them equipment and who tried to help should 
be noted.) When the representative from SIGI drew the equipment 
failure to the attention of the principal investigator and asked 
what to do, the principal investigator suggested chat SIGI could 
show its user materials and handbooks. At that point the 
representative from SIGI said they had brought none. (The need 
for user materials had been specified in the letter of 
invitation. ) While machines can fail in the best planned 
circu I tances, SIGI seemed to be experiencing several 
difficulties of organization the day of the demonstration. SIGI 
withdrew from the demonstration. 

Accordingly five systems were presented orally and 
demonstrated to all members of the System Review Panel as 
described in the methodology. The panel rated the systems on 15 
criteria developed from the findings of the previous surveys 
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conducted. Each criterion was rated according to the following 
scale: 

5 - Excellent 

4 - Good 

3 - Fair 

2 - Poor 

1 - Atrocious. 

Examination of the item means for each system show that, in 
general, the System Review Panel found the systems between "fair" 
and "good." Of the 75 item means, 40 fell between 3.0 and 4.0. 
Only two item means fell between "poor" and "fair." The 
remaining 33 fell between "good" and "excellent." Table 22, 
System Review Panel Ratings of CIDS: Unweighted Item Means, 
shows the results of the computation of unweighted item means 
and standard deviations. The item designations have been 
abbreviated in the table and can be seen fully in Appendix VI. 

In examining the item means and the total scores, some 
limitations of the system review process should be borne in mind. 
First, the time to view the systems was restricted. Second, 
there were several comments about the unevenness of the quality 
of the presentations by CIDS representatives. Third, some raters 
may have been influenced, either for or against a system, by 
prior experience with that CIDS. 

To determine the total System Review Panel rating of each 
CIDS five of the items in the table were weighted twice the value 
of the others. They were: 

3. Appeal of the system — the extent to which it 
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encourages client or student use; 

12. Ability of users to enC:er the system at their points of 
readiness or interest rather than at system-determined entry 
points; 

13. Ability to use the system in group or classroom 
settings ; 

14. Overall ease of use; and 

15. Overall rating. 

li'ie total results of the rating of the System Review Panel 
for eaa'i' CIDS is presented in Table 23, System Panel Review 
Ratings of CIDS: Total Weighted Score. The order of the CIDS on 
this rating was CHOICES, CIS, Discover, C-LECT, and CIS. 
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Table 22 

SYSTEM REVIEW PANEL RATING OF CIDS: UNWEIGHTED ITEM MEANS 



Item 


C' 


-LECT 


CHOICES 


CIS 




Discover 


GIS 




Reading ease 


3 


.38( 


.82) 


4.79(.48) 


3 


» r > i 


.94) 


4.36( 


.54) 


4. 


.03( 


.72) 


Screen clarity 


3, 


.24( 


.92) 


4.85( .36) 


3 


.21(1.01) 


4.42( 


.60) 


3. 


.91( 


.71) 


System appeal 


3, 


.33( 


.92) 


4.55( .61) 


3 


.03(1.00) 


4.21( 


.54) 


3. 


.97( 


.76) 


User handbook 


2. 


.88(1.53) 


4.40(.95) 


3 


.33(. 


.97) 


3.18(1.36) 


4. 


12( 


.91) 


Search of occ 's 


3. 


,88( 


.69) 


4.39( .60) 


3 


.42(1.05) 


4.24( 


.78) 


4. 


12( 


.81) 


Movement in sys 


4. 


,24( 


.60) 


4.12(.69) 


3, 


.79(. 


.95) 


4.49(, 


.70) 


4. 


30( 


.76) 


Occ info detail 


3. 


82( 


.83) 


3.85(1.10) 


3, 


.64(. 


.95) 


4.18( 


.80) 


4. 


24 ( 


.74) 


Terminology 


3. 


76 ( 


.74) 


3.97(1.09) 


3. 


.67(. 


77) 


3.79(. 


.73) 


3. 


70(1.17) 


Occ to ed links 


3. 


76( 


.74) 


3.82(1.00) 


3. 


.61(. 


95) 


3.70(, 


.83 


3. 


94( 


.86) 


College search 


3. 


88 ( 


.88) 


3.91(1.05) 


3. 


,61(. 


98) 


4.06(. 


.69) 


4. 


52( 


.66) 


College info 


3. 


79( 


.84) 


3.49(1.35) 


3. 


,70(. 


87) 


4.27(. 


.90) 


4. 


61( 


.69) 


System entry 


4. 


21(. 


.73) 


4.12(.75) 


3. 


61(. 


99) 


4.27(. 


71) 


4. 


09 ( , 


.9/) 


Classroom use 


3. 


12(. 


.81 ^ 


4.06(.89) 


2. 


97(. 


94) 


3.58( . 


92) 


3. 


85 ( , 


.96) 


Ease of use 


•J 


61(. 


.74) 


4.24( .65) 


3. 


24(1 


.02) 


4.21(. 


59) 


4. 


09 ( . 


.90) 


Overall rating 


3. 


70(. 


,63) 


4.33(.64) 


3. 


18(. 


91) 


4.12(. 


73) 


4. 


18 ( . 


.63) 
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SYSTEM REVIEW PANEL RATING OF CIDS: TOTAL WEIGHTED SCORE 
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Conclusions and Final Ratings of the CIDs 

The major purpose of this study was to identify one or more 
Career Information Delivery Systems that would meet the needs of 
New York State in implementing the occupational education 
curricula and guidance plans and serving populations making 
career plans and decisions. 

The first surveys conducted, TiVe Information Validation 
Survey and the Surveys of User Satisfaction along with the site 
visits yielded a base of information upon which the rating of 
systems could be carried out. This base of information was 
translated into three concrete measures of systems: 

-information from vendors about system components and 
features supplied in the response to the RFI; 

-an expert analysis of information on an occupation and 
college; and 

-a system revlev '-^ interested and knowledgeable 
professionals. 

Each of the measures has its strengths and weaknesses. Tiie 
response to the RFI, for example, depends upon the vendors' 
interpretations of the questions and their assessments of their 
systems. The expert analysis of Information, on the other hand, 
depends not only upon the knowledge of the analyst, but on 
whether an analyst rates high or low. The rating of the System 
Review Panel may be influenced by the style of the presentation 
or previous experience with a system. 

Taken together, however, the three measures balance each 
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other and serve as stronger predictors of satisfaction with a 
system than any one of the measures alone. Therefore, using a 
basis of 100 points for each of the three measures identified 
above a final score was computed for each system. This gives 
equal weight to each of the measures. The possible 300 points of 
this score is then expressed as a percentage for a final system 
rating. 

The final scores for each of the CIDS evaluated are listed 
in Table 24, Final Rating of CIDS: Summation of all Factors. 
In the table, the systems are presented in descending order of 
their final scores: CIS, GIS, CHOICES, Discover, and C-LECT. 

It may be noted that, while specific strengths and 
weaknesses of the systems emerged in the findings of this study, 
four of the systems are very close in overall score. Second, 
none of the systems has a spectacularly high score. Finally, the 
overall scores of the systems seem to reflect the only moderate 
level of satisfaction that counselors expressed in their 
responses to the Survey of User Satisfaction. 
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Table 24 

FINAL RATING OF CIDS: SUMMATION OF ALL FACTORS 

Rank System Contents Information Review Panel Final Score 
. Accuracy 



1 CIS 94.00 65.26 67.33 75.53 

2 CIS 68.00 71.58 81.85 7,3.81 

3 CHOICES 67.34 67.37 84.24 72.98 

4 Discover 62.00 67.37 81.48 70.28 

5 C-LECT 79.00 43.16 72.76 64.97 
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Chapter Five 
ANCILLARY ISSUES 

During the course of this study several issues emerged which 
had not been anticipated in the original plan, nor identified in 
the earlier request for proposal from the State. These issues, 
and the events related to them, are discussed in this chapter. 
The issues are: 

-compliance of the CIDS with civil rights legislation; 
-communication between the State Education Department and 
local people responsible or securing and using computer- 
based career information delivery systems; 
-the relationship between planning for CIDS and other 
computer planning at the State Education Department; and 
-the emergence of related computer-bfe' d systems. 

Compliance of the CIDS with Civil Rights Legislatici 

During the study the investigators learned that questions 
regarding the compliance of C-LECT with civil rignts legislation 
and court decisions had been raised by the Office of Occupational 
Education Civil Rights Coordinating Unit (OECRCU) . These 
questions had come about as a result of a routine compliance 
visit to a school district using the system. 

To learn more about the matter, a meeting was initiated by 
the principal and associate investigators with Dr. Karl Wittman, 
Program Manager. In the meeting, it was learned that after the 
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visit to the school district » C-LECT had been further reviewed, 
at their request by members of the OECRCU, and that they were in 
the process of correcting the problems identified. It was also 
learned that no other CIDS had been thoroughly reviewed fc he 
OECRCU . 

Dr. Wittman felt that understanding of C-LECT had been 
facilitated by the fact chat it was demonstrated by its 
representatives directly to the OECRCU. It was, therefore > 
decided to take advantage of the opportunity provided by the 
vendors being in Albany for the system demonstration. Each 
vendor would present the CIDS to a team from the OECRCU. Since 
the demonstration day was to be very full of activities, it was 
decided to use the days immediately preceding and following June 
24th for the civil rights review. It was understood that this 
review would be carried out by members of Dr. Wittman 's staff 
using their criteria for civil rights compliance, and would not 
take place during the system demonstration on June 24th nor deal 
with the same issues. 

Since th_s activity had not been p t of the original study, 
it was reviewed y Barbara Shay, design ;d liaison to the State 
Education Department and a member of ; "ject oversight 
committee, who approved it. Accordingly, along m th the Request 
for Information and i /itation to participate it: the System 
Review Panel demonstration, the vendors were sent information 
regarding the planned review by compliance unit. Each system was 
asked to send its user's manual, counselor's manual, descriptive 
brochure, and, optionally, a description of any components 
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designed specifically for use with special populations to he 
OECRCU. 

It was understood by Dr. WiCtman that the report of the 
OECRCU would becomt^ a part of this report. That report has not 
been received. An utiofficial conversation with a member of the 
OECRCU indicated that no problems had been identified with any of 
the systems, and correspondence froDi C-LECT indicates chat any 
problems had been rectified. 

Communication Between the State and Local Users 

Some concern was expressed by people in local educational 
agencies and regional computing centers that they had not been 
aiade aware of the study. The sense they communicated was that 
not only were they not aware of this st^udy, but information on 
actions by the State Education Department which should have an 
effect on their selection or use of CIDS was not always 
available, nor was It available in a tin.ely manner. 

In the case c.i this study, which had very short timelines, 
th* people who were aware of the study at all learned about it as 
it was happening. However, the concerns expressed do have 
implications for iucure actions. 

There are at least two major groups within the State 
educational community which should be informed of the results of 
the CIDS evaluation and of the actions the State will be taking 
as an outcome of the study. One group is the network of people 
from the regional computing centers. They seem to be the key 
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players in decisions regarding the purchase or lease of computer 
software. Certainly, the State would benefit from coordinated 
activities of the centers In purchasing. To some extent, this 
already takes place In the leasing of GIS. 

Durln.^ this study, a visit wa*- made to the Onondaga-Madison 
BOCES regional computing center in Syracuse to discuss the role 
of computing centers in distributing CIDS. There are 14 regional 
computing centers throughout the State serving the BOCES and many 
of the school systems iv. theii regions. The centers deliver on- 
line and microcomputer software to schools. 

Presently, CIDS vendors deal directly with these centers and 
the BOCES. When a school orders a CIDS, the product is delivered 
through the centers back tc the school. Each center has a person 
OS staff designated as a trainer for the systems. It appears 
•-:ha^ :hes.e people are basically computer experts who receive 
trfiinirij on CIDS from the vendors and, in turn, train the 
counselors. These syccem trainers should have been ideni;ified 
early in this study and been involved in the system review 
process. 

It should be noted that seme school system? do not e the 
services provided by the regional computing centers. Ir>tead., 
they obtain, deliver, and arrange for training on systems 
Independently. This situatir;n underscores the need for 
coordination and communication between the State and local users. 

'^he second group is guidance counselors. They are the most 
iniiuential people in the actual use of the systems. It is 
recommended that information about CmH ha disscaiinated to these 
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groups through special staff development workshops or through 
presentations at their regularly held meetings. 

Computer Planning 

In the State Education Department, planning is currently in 
progress for a model of computer use which combines distributed 
cw.pu^;:-:g with the ability to have data input to powerful small, 
locc. . computers from a large, regional databf, e depository (A 
Primer in the Methodology of Systems Analysis for Application in 
Categorically Aided and Other State Programs, 1985.) 

If the State were to choose to enter its own data in the 
CIDS being used, it would seem desirable to investigate the 
feasibility of using this network to do so. Possible connections 
should be explored. 

Related Computer-Based Systems 

Duv.'ng the course of che study other computer-based systefli^^ 
which assist people in some phases of career planning were 
brought to the attention of the investigators. These included 
the Ability Information System being used by the Worker Re- 
employment Center of the Buffalo and Erie Private Industry 
Council, Inc.; an interactive video disk system. College USA; and 
a similar system called. Passport to Your Future. While none of 
these systems were identified for inclusion in the study, and 
they are different in scope and focus from the computerized 
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career information systems studi«.J, they might be of interest to 
a school or agency considering the use of computers to provide 
career information. In some cases they might be seen as 
additions to a CIDS. 

In addition to the systems named above, other systems known 
to the investigators were not included in the study because they 
were not in use in New York State, in particular, the 
computerized version of COIN. 

Finally, it is clear that more and more systems will emerge 
in the competitive environment that exists in all areas of school 
sales. HVerefore, the study recommendations will include 
provisions for examining and rating the quality of other systems. 
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Chapter Six 
SUMMARY AND RECOMMENDATIONS 

Summary 

The purpose of this chapter is to present '-he long view of 
what the findings of the study tell us aw York State should do 
to provide computerized career information to the schools 
implementing the occupaticual education curricula and guidance 
plans as well as the schools and agencies serving all the 
populations involved in the various stages of career planning. 

At the inception of the study five questions wtre asked: 
The first question was: "What is needed in a computerized system 
to meet the career information needs identified for New York 
State?" The second question was: "What systems are already being 
used in the State, how effective and well liked are they by thPir 
users, and what gaps are there in thoss systems?" The thi 
question was: "What can vendors of systems offer New York 
to help mr-et its needs?" Fourth: "How much will that cost i. 
State, comparatively?" The final, and ultinate, question was: 
"Given the answers to the four questions, what steps should the 
State take to adopt or develop one or more career information 
systems to provide national and State information to the broad- 
based user community?" A series of activities were carried out 
to answer the questions. 

The first question was answered through a review of the 
literature which yielded a list of 85 information components and 
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design considerations which were validat&u ..hr- uht: responses 
of the national and state panels of experts to rh; Information 
Validation Survey. 

Surveys of User Satisfaction of principals, counselors, 
students, and parents confirmed and enlarged the list. In 
general, these surveys showed that the CIDS were being used in 
the schools primarily for guidance with some classroom and group 
instruction activities. The students' responses showed that they 
found the systems easy and enjoyable to use and more useful for 
information gathering than for career decision making, thus, 
indirectly, supporting the idea that a career information 
delivery system is a resource to be used as part of an overall 
career guidance or education program. The jarents' response's 
showed that they strongly supported the use of computerized 
systems in their schools and the Mse of computers for career 
information delivery and exploration, but not for values 
clarification. Counselors were highly supportive of computerized 
career information, but only moderately satisfied with the 
systems they had. Counselors expressed particular concerns with 
the quality and timeliness of the information in the systems and 
the adequacy of the training. From these surveys and the site 
visits, strengths and weaknesses of the systems in use were 
identified and the second question was answered. 

The third question, \,?hat can the vendors offer New York 
State, T„af approached several ways. Five of the six major CIDS 
in use in New York State responded to the lengthy Request for 
Information, provided printouts of occupational and educational 
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which was analyzed for comprehensiveness and accuracy, and 
demonstrated their systems to a review panel. A process for 
rating the CIDS taking all of this into consideration was 
developed and used for measuring the part and total value of each 
system, while different systems excelled or were weak in 
different aspects - information components contained, system 
design considerations, user support materials, information 
comprehensiveness, rating by the System Review Panel the final 
analysis showed that four of the systems rated satisfactory (in 
descending order: CIS, GIS, CHOICES and Discover) and one (C- 
LECT) just passing. This confirmed the counselors' levels of 
-atisfaction, or dissatisfaction, with the CIDS in use. It can 
be said that all of the CIDS showed room for improvement. 

While costs were not analyzed as part of the total rating 
system, they were gathered through the RFI and comparative 
analyses presented. The analysis showed large variations in 
basic system delivery, developmental, and total costs. It ^ilso 
revealed important differences among the CIDS in such issues as 
entry and ownership of state developed data and files. 

While the study was in progress, other issues came to the 
fore. They included: issues related to the ccjipliance of the 
CIDS with civil rights legislation and judicial decisions; the 
need for communication with CIDS system trainers in the regional 
computer centers and with guidance counselors; the possible links 
that can be made with other computer planning taking place In the 
State Education Department; and the continuing emergence of new 
computerized career information delivery systems. 



Er|c 187 



CIDS Evaluation, 175 
The findings of the activities led to sore answers to the final 
question. These answers are presented as the recxitmendations which follow. 

Reccninendations 

liie recatitendaticns take into consideration a nvinter of factors. 
First, there was a high degree of general support for ccrputerized career 
informaticn delivery systons expressed by the various user camiunities . 
Second, although there were weaknesses identifi.;.? > die systems, there was 
g'?De,caily more satisfaction than dissatisfactia , ;j diere has been a 
varyiiig degree of investment of resoiirces in each or the CIDS by schools and 
agencies, within the State. Third, the greatest area of dissatisfaction 
could be interpreted as a hue and cry for more accurate, more up-to-date, 
and more specific occupational and educational information. Fourth an 
analogy to the tradition in New York State for approved, rather than adopted 
texts, may be drawn to educational software, and thence to CIDS. ihere are 
eight reccninendations. 

1. Ihe State should pursue the development and utilization of career 
information delivery systems. 

2. No single system should be adopted, instead, s-/stems should be rated 
and,, if satisfactory, approved. 

3. Sane meaning should.ie ascribed to the feet that a systems is approved. 

Perhaps use of those systems could be eligible for VEh funding or other 
assistance. 

4. The five systems rated for the study (C-LBCT, CHOICE? CIS, Discover, 
and GIS) should be approved. (SIGI, \Anjch did not carplete the process, and 
any other systt.© should not be considered approved until they are rated.) 

5. Hie process for system approval and the related rating sheets presented 
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in Appendix VII, Procedure and Instruments for Evaluating CIDS, should .be 
adopted as the process for approving systeniSo This process includes an 
evaluaticp? of information si;5)plied by the vendor, an analysis of tlie 
ccnprehen:-iv»ness and accuracy of information supplied on the vendor's 
sairple ; ^. ts, and a rating of an actual demonstration by iittfirCiited and 
kncxcledgeable x:isers. 

6. Activities such as staff developnient workshops and printed connunica- 
tions should be undertateai as soon as possible to inform those responsible 
for buying and using CIDS of the results of this stxjdy, particularly of 
which systems are approved and of how to rate other systans. Hiis user 
comiunity inclucJes: school and school district aaninistrators , : ^ency 
heads, counselors, regional center coordinators, and appropriate profes- 
sionals in the State Education Department. 

7. In view of the strong need for more accarate, up-to-date and specific 
occupatic^al and educational information, sufficient resources must be 
providec ..y New York State to its agencies to establish or upgrade the 

r Jinary data series needed for the CIDS. Although a great deal of tnfoima- 
tion is already collected by the^e agencies, they do not have as tiieir 
primary audience the user ooninunity that the CIDS serves. The iiiformation 
validation survey and the. surveys of.usa: satisfaction give strcng indica- 
tions of information oonponents needed and the level of detail appropriate 
to the us6r audience. 

8. Hie function of coordinating and continuously inprovinq career informa- 
tion delivery should be focused in one location. A unit, the Cax^j^x Infor- 
mation Delivery System Coordinating Office (CIDSOO) should be created in the 
State Education Department v*iere a strong network of xisers already e:adsts. 
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Six functions are reccninended fcr the CIDSCO: 

A. The CIDSCO should develop state career infontatiOT incliading 
infonnaticn about: occupations, secondary areas of concentration, post- 
secondaiy educational programs, colleges and universities, proprietary 
schools, financial aids, and scholarships. It must be stressed that a 
separate unit be responsible for this although a great deal of the informa- 
tion is alreat^ collected by the responsible State agencies because these 
agencies do not have as their primary audience the xiser catmxT.ity which the 
CIDS serve. In sane cases the collected data needs to be clur:tered because 
it is too detailed for an individJietl's use. In other cases,- :eeded infonna- 
tion is not collected. For exanple, the Office of Higher o-i Professional 
Educations was not able to sv?>ply information on intercolle-iate sports at a 
SUMY school. Ht-^xrver, this information is considered very inportant by 
scBie, particiiloi those dealing with inner-city youth. The CIDSCO wotild, 
therefore, v^ tk ix- oonjuncti ^i with other agencies responsible for primary 
data collection, 

B. The CIDSCO should be responsible for seeing that the information 
is incorporated in the approved CEDS. Uiere are two options for doina this. 

It can be entered by tiy^' - ate or turned over to vendors for entry. It is 
strongly reooonwnded that the State enter this data itself despite the fact 
that two of the approved systems do not currently allow this. The state 
should negotiate this ri^t with a one-time fee to those systems alcng 
with aavuranoes that no inappropriate changes in the database will be made. 

There are two reasons for the State's entering, as well as developing, the 
data: first, there will be greater control over the accuracy and timelines 
of information delivery; second, interpretations of information that will 
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have to be made will be made cansis1;eritly across all systans. (It should be 
noted that entering data oneself is not free. While money will not have to 
be paid to vendors, it will have to be paid to people entering the data.) 

C. The CIDSaO should provide ccnputer expertise in CIDS management 
and development, sane systans are narrative-based, others attribute-based. 
Most use sane form of data crunching. Software will have to be developed 
and refined to accatmodate the data entry. Where possikils, this should be 
coordinated with other units planning conput . tworxs in the State. In 
addition > as new technology is developed, CIDS will be available to take 
advantage of it. The CIDSCX) should be in the forefront of identifying ne^/ 
developments, in particular conputer technology that will enable people ath 
handicapping oonditions to get career information. 

D. Sines we clearly have not seen the end of new CIDS on the market, 
the CIDSOO should be responsible for rating and approving other CIDS using 
the process outlined in Appendix VII. 

E. Hie CIDSOO should negotiate procurement «ntracts with all 
apprcned CIDS so that all scdx»ols and agencies in the State are diarged 
uniformly for the same products and services and so that the State may take 
advant^age of bulk purchasing rates. Hbere is saae model for this in the 
State with GTS aM the Cttcndaga-Madison BOCES as well as with CI 

New York Board of Education. 

F. Ihe sixth function of the CIDSOO is informe d ion dissenir^tion and 
staff development regarding CIDS. Hiitially, it is reccrmended that they be 
responsible for letting schools and agencies know which systems are approved 
and how to get e^pproval for other CIDS. In addition, while traiiiing is 
provided by vendors in the uses of their systems, the CIDSCO should provide 
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staff developnent to leaders in the occupational education and gtiidance 
ccnmanities on the general uses of CIDS to iitplenent curricula a«a guidance 
plans. 

Ctoviously, not all of these reccnmendations can be carxied out imtEdi- 
ately. However, there are first steps that can be taken vtdJ.-?: funding for 
the CIDSC30 is sou^t and the unit is established. First, the findings ^f 
this study can be disseminated to a broad base of occupational education, 
gxiidance, and agency personnel. Second, priorities of infonnation develop- 
ment can be established based on this study, the work of CIDS in other 
states, and the availability in occupational and educational infonnation in 
New York State. 

Wi.th ti-ie establishment of the CIDSCO, New York will take the best 
possible ^roach for serving CIDS users across the state. First, bj' 
providing accurate and xj?>-to-date career infonnation to a wide range of 
users, the state will be performing a much needed service, one that will not 
only be of value to its individual citizens, but that will contribute to the 
e-oncndc health of the State and. its regions. Sftcctiji, while mainvadning 
quality control of the infonnation and setting standirds for delivery, the 
State will be supportivB of local decisicns to raeet local needs and piefer- 
ehces. Ttlis Innovative apprbach to ocrputerlzed career .Infomv^tion delivery 
is particularly appropriate to tlie size, d^.versity, and resources of New 
York state. 
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INFORMATION VALIDATION SURVEY; 
INSTRUMENT AND RELATED MATERIALS 

Information Validation Survev 

Memorandum to New York State and National Experts in Career 
Guidance 

Memorandum to Major Providers of Occupational Education 
Excerpts from Three New York Occupational Curricula Guides 
Excerpts from "District Guidance Plan: Implementing 
Commissioner's Regulation 100.2 (j)" 
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INFORMATION VALIDATION SURVEY 



FULL NAME : 

TITLE 

INSTITUTION 

BUSINESS ADDRESS . 

CITY STATE ZIP. 

BUSINESS TELEPHONE 

SURVEY DIRECTIONS 



Th# purpose of this survey is to validate information and 
syst#m n##ds for a computerised career information delivery 
system (CIDS) to serve a broad*range of users* Th#se needs 
include! (1> to help implement three occupational education 
curricular proprams Cbgmt tQi C«C££C S^illSt lQl£.£dU£liSD Ifi 
It^bDSlfiflyv «nd IntC£dU£iiS:;Q l£ Q££UC££i&Q8> New York State; 
(2) to complement implementation of guidance plans; (3) to assist 
career and educational planning by individuals in such groups as 
junior and senior high school studentSf out-of-school youth,* 
community and technical college students^ handicapped workers, 
and other adults. 



Please consider each of the components on the survey in light 
of the needs listed above. (Excerpts from the curricula and the 
guidance plan guidelines are attached for your information.) It 
is important to note that the curricula are mandated for students 
in seventh, eighth, and ninth grades. 

The 85 items on the survey are divided into three categories: 
Occupational Information, Education and Training Information, and 
System Design Considerations. The items in each category have 
been identified as the result of a review of the literature and an 
analysis of the career information needs for the curricula and 
guidance pl£n guidelines. Next to each item in each category is a 
scale from 5 to 1, with "S" representing *'Highly Desirable, 
definitely include in a computerized career information delivery 
system** a:^d "I** representing "Highly Undesirable, definitely do 
not include in a computerized career information delivery system**. 

Please complete the identifying information at the top of 
this page and place your name at the top of each subsequent page. 

SURVEY 

I. OCCUPATIONAL INFORMATION 

A. Infornation Components Related to Specific Occupations 

Niglklr Mtlthvr DMirafelt Kachljr 



1. OccflpitioMli \i\U with idtfitifying cede 9 4 3 

2. flcci^^UoMl ituri^tioii 9 4 3 

3. OccnMtioul groif or dttittr 9 4 3 

4. ftttUs 9 4 3 
9. Iatffrtit» vorktri u vilidatttf in virioits studiti) 9 4 3 



7. TtipffriMati 

I. atry Uvtl sUUi rtqoirtd for job 



S. Plyiicil dftMRdt (atmX of Mig^.t tbt wrkw tost lift or carry) 9 

10. Mtyiicol otiiviiiti 9 

11. Tools, tqaipMit, fttc. astd 00 tilt job 9 

12. EovirooMiUl/oork centfttioos _ _ 9 
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3 
3 
3 
3 
3 
3 
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INTORMATION VALIDATION SURVEY NAME 



13. Hoari of mi «ikd trivtl r^uirH m tht job 

U. Induitriff iR vkich occupatiMi occir (placti of toployMut) 

IS. HiriH cMfwtlt 

U. Urttr lidtftri »i advanctimt opporUnititi 
17. RccttMtiiMul ovtlook 

11. CirrMt Md futort occupational supply and dtMRd 

19. CvrMt nployMiit (wiaberi, fty occupation by induitry) 

2tf. £irniR9i and btntfiti (btgintiieQ, avtrage, rm^i, 
fringt btntfiti) 

21. Cducatiofi/traiRing rt^uirtd 

22. UcMiiRp, ctrtificatiofi, tuts, ttc. rtquirtd 

23. M^iut Mcoodary Khool Mibjtcti 

24. Stiattd occupaticni 

25. Rtlattd tilitary occupitiont 

2fi. Seorcfi of additiofUil inforutioa 



s 
s 
s 
s 
s 
s 
s 

s 
s 
s 
s 

9 



B. Information Components R*Iat«d to Labor Mai 



27. Job iMking sJcilli 

2S. Stlf-Hployifiit and tntrtprMtitral inforiation 



II. EDUCATIONAL AND TRAINING INFORMATION 

A. Information Components Related to Secondai 
Concentration 



2). ArM of conctfitration deicriptioo 

30. ArH of coocMtratioo froop or clotttr 

31. Typical coortt vork 

32. Typts of ItarRiif activitits 

33. SkUIt Md iMMlfdge icquirtd 
ex. Rflattd Mtry 1ml occopations 

35* ItUttd occopationB rcqtiirint f«rtKtr itvdy 
%* Milfi poft^^mdarr prograas 



S 
9 
5 
5 
5 
S 
9 
3 



3 
3 
3 
3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 



ket Entry 

3 
3 



Areas of 

3 
3 
3 
3 
3 
3 

3 . 
3 



B. Information Components Related to Post-Secondary Educational 
and Training Programs 



EKLC 



37. ProgrM dtscri^ioo 
38« Profraa groHp or dttsttr 

Typiul conrsf vork , 
40. TypH of iHTilti ictifitiH 
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XHTORtlATION VALIDATION SURVEY NAME 



NifMr 



41. Skini ni toovlHgt Kqvirtd 

42. Ctrtificitts or iigrcts t«rK*4 

43. ioTiil l«fi|th of liM to coaplfftff Cfrtifiutf or tf^grof 

44. IfUtod xc#4tioii 

45. IffUttd K09rMS of itttdy 

44. Tyyid) of liititvtioiii offfriif ^rogrM 



C. InforMtion CoApon^nts R«l«t*d to 
Institutions 



47. Ruf Md addriii of tducatiofial or training imtitution 

48, (t«9r«phic location 
4). Typff of inititutioR 

50. Adiiiiion Rt^BirHinti (tut Aom, application proctdsrfii 

application diadlini, fm, ^^'^ical iiu, ttc.) 

51. Raof and addriii of adaiiiioni contact pirton 

52. Cost (in-statff and oat-of itatf tuition, root, board, fm) 

93. Hooting (availability, ofl-taipus, off-caopui, pirc^nt of 

ndirgraduatH living on caapus, rtqairid of frffhetn, itc.) 

94. financial aid infornation (tourcii, application procidurfi, 

appUcatioo diadlinf, aviragi award, ftc.) 

99. Mast asid addriti of financial aid contact person 

96. Education and training progrws offirtd 

97. Typii of cirtificatffi or digrffi mrdtd 

56« lorial Itngth of tiM to cetpltti cffrtificatfi/degrfffi offirid 

9). Spicial prograoi availabli (coopirativi tducation, adult baiic 
«ducation, foriign itudy, ROTC, ttc.) 

U. Library and Ifarning cintfr facilitits inforoation 

(1. Sippori Hrvicffi availabli (couniiling, liarning lab, placinint) 

&2. StrviCH for bandicapptd (koutiog and building acciii, 
dffiignatffd parking, intirprftiri for thi diaf, tutori, 
liarning aidi, couniiling, ngistration aiiiitanci, 
transportation for thi tobility iipairid) 

S3, lotircolligiati sports offirid (by sti and availability of 
scholarships) 

M. kcriditation infortation 



Educational and 



^sr copy AVAiMBu" 



D. Information Components R#Iat#d to 
Aid and Scholarships 



tt. eligibility riquirtiints 

87, Application prociduris and diadlini 

M. Siti or basis of avard 

Uatf snd addriss of contact pirion 
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Ihff-ORMATION VALIDATION SURVEY NAME 

Swoirau btwrai* Nir UnMwrait UuMtirait UnMrai* 

70. Nov to obtain «dditioii4l inforutioft 9 4^21 

XX Z. SYSTEfl DESXGN C0NSXDERA7X0NS 

71* IntffrKtivt ttriKtuTttf M4rck of occupitions by uch tlfttnti n 
iitntitii tttptrMtntii u\M, Md ikiUi prtviously Kquirtd 

72. Ability for ttSffrt to qutry tht tyttti n to vfcy sptcific 
occuy^tioni did lot M^pur m tbiir liiti 

73. DirffCt accfii to ioforutioo abo«t occup^tioni 

74* lirffct ACCffii to iflforiatioo ibout iKOodAry trta of 
coocffltritioti 

73. DirKt iccffii to inforMtion about post-iKondary fducationil 
wd training progrm 

7&« Intfractivf itructurtd ttarch of tducational inititutioni 

77* Oirtct accffis to inforMtion about ffd»'::t!&nal inititutioni 

78. Ability to utch inititutioni and tducational prograis 

71 Linki to publishtd asifftiotnt initruaints 

10. Ability to uiff occupational charactfriitici to sort for 
tiiilar otcupationi 

81. Intfraclivf ttructurfd iiarch of financial aid intonation 

82. Ability of thf lyitM to opiratt on a licrocooputir 

83. Ability of diinti to vaU aviy froo a itiiion at tht coiputtr 

vith a printffd ricord of tiarchfi and inforution rtctivtd 

84. Ability to add'itatf spKific occupational and educational 

inforution to tziiting rtcordi 

83. Ability to criatf rtcordi for nev and fifrging occupationi 
Md training progrui 

Limt b*lowf by category, any additions 
on the survey* Attach additional page 
X-A 

X-B 

II-A 

XX-B 

XX-C 

XXX 



to or comments on the 
if necessary. 



Items 



PLEASE RETURN YOUR COMPLETED SURVEY BEFORE APRIL 30, 19Q6 TO: 
Dr. Deborah P. Bloch 
Box 20369 

Cherokee Station Post Office 
^^Y^. O i -Mw^w New York, New York 10028-9391 

ERIC^) COPY AVAILABLE 
— ^ 2M 
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Deborah fcrfmutterBlcx:h,Ph.D 



MEMORANDUM 



TO: 



New York State and National Experts in Career Guidance 



FROM: 



Deborah Perl muster Bloch 
Joyce Ford Kinnison 



RE: 



Information Validation Survey 



We appreciate your willingness to complete the attached 
Information Validation Survey, The Information you will provide 
is essential to the study currently being conducted to determine 
the career information delivery system best suited to meet the 
needs of the people of New York State. 

One activity in this study is the identification of the 
information needs of the many groups and individuals whom the 
system will serve. To accomplish this# career information 
components and system design considerations have been identified 
through an extensive review of the literature and an analysis of 
the new Occupational Education Curricula and the requirements for 
district guidance plans for the State, 

The purpose cf this survey is to validate the findings of 
the review and the analyses^ and to see whether experts in the 
field find some of those components more desirable than others. 

For your information^ selected materials from the curricula 
guides and the guidance plan guidelines are enclosed with the 
survey. Please return only the completed survey before April 30# 
1986. An envelope is enclosed for your convenience. 

If you have questions^ call Dr. Bloch at (212) 794-1098 or 
Ms Kinnison at (919) 469-0081. 

Thank you for your cooperation. 
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Deborah Berim tterBloch,Ph.D 



MEMORANDUM 



TO: 



Major providers of occupational education 



FROM: 



Deborah Perlmutter Bloch 
Joyce Ford Kinnison 



RE: 



Information Validation Survey 



The attachf^d survey is to be completed by someone in your 
organization who is familiar with the career information needs of 
the students you serve. This could be a Director of Guidance, a 
Director of Occupational Education Programs, a counselor, an 
occupational education teacher, or an administrator. We depend 
on you to select the person best qualified to complete the 
survey. 

This survey is part of a study currently being conducted to 
determine the career information delivery system best able to 
meet the needs pf the people of New York State. A letter 
which explains the study from James A. Kadamus, Assistant 
Commissioner for Occupational and Continuing Education, New York 
State Education Department, is attached. 

Please give the survey and the enclosed materials to the 
person you select, and ask them to return only the completed 
survey before April 30, 1986. An envelope is enclosed for your 
convenience . 

Thank you very much for your cooperation. 
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EXCERPTS FROM THREE NEW NEW YORK OCCUPATIONAL CURRICULA GUIDES 



The three occupational education curricular programs 
examined here are being pilot tested in New York State during the 
1985-86 school year. They are: Home and Career Skills for 




1 n 



Even though the structure of the programs may change as a 
result of the pilot tests, occupational information will still be 
needed to meet the overall goals of the program. Therefore, 
order to present the broadest picture possible, the following 
discussion about data elements required to compliment the 
programs includes both general analysis of needs and specifir 
needed data. 

Hoine and Qangfti: Skills 

There are three broad objectives for this program, which is 
mandated for all students in the seventh and eighth grades. The 
first is to "Develop skills which lead to effective decision 
making, problem solving, and management in the home, school/ 
community, and workplace" (page 4). 

One of the areas to which the principles and process skills 
are 'o be applied in teaching this program is: 

Career Planning— an overview of work, tentative plans, and 
entrepreneurship.. .Career Planning gives students the chance 
to begin making decisions and solving problems related to 
tentative career plans (page 4). 

The ^Career Planning' segment of the course "provides an 
opportunity to anticipate the future, identify employable traits, 
examine oneself in relation to the work environment, and to 
explore career opportunities". It further asserts that 
"Application of the decision making process encourages studa-nts 
to form tentative career plans" (page 114). 

Three of the seven competencies to be achieved are: 

2. Identify personal characteristics which lead to job 
satisfaction 

3. Formulate personal criteria for career planning 

4. Apply the process skills to develop tentative career 
plans (page 115). 

Scattered throughout this 'Career Planning' section are 
activities which require the student to use occupational and 
career information to meet goals. Seven examples of these are: 

<1) 3. Describe characteristics of work involving various 
work locations and conditions, work personnel, and work time 
or hours. . .Several work characteristics/environments are 
suggested. . . " 



You would like to work: 
-for people caring and serving 
Q -with data, facts, and records 
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-with ideas, theories, and thought 
-with things, machines, and tools 
-outside most of the time 
-inside most of the time 
-with other people 

-as "boss" with responsibility for others 
-following directions given by others 
-where you'd get dirty 
-where there's lots of noise 
-where it is quiet and calm 

-where you have many decisions to make doing physical labor 
-wher e the wor k woul d be danger ous 
-where you would help others 
-where you earn lots of money 

-where you earn lots of money doing something you don't like 

-designing and creating things 

-doing the same thi ng over and over 

-at home on a flexible schedule 

-wf;ere you need to take work home 

-on a f 1 exible schedul e 

-on "off peak" hours Cpage 118) 

(2) 1. Given related checklists and evaluation aids, the 
student will apply the processes of decisionmaking, problem 
solving and management to personal career planning, with a 
degree of understanding acceptable to the instructor. 

In order to do this, the student must be able to: 

a. Identify areas of personal interests, abilities, and 
aptitudes 

b. Identi fy personal skills (mental, physical, emotional, 
soc ial > 

c. Identify personal attitudes an values toward work, 
training/education, and life styles 

d. List home, school, and work experiences which contribute 
to career selection 

e. Formulate criteria for personal career planning (page 122) 

(3) 2. After preparing criteria for personal career 
planning, the student will use the decisionmaking process to 
formulate tentative career plans... 

In order to do this, the student must be able to: 

a. Identify and use available career resources to obtain 
information about careers and employment trends 

b. Identify career clusters and opportunities for the 
development of transferable skills 

c. Match career character i:?tics and personal characteristics 
to serve as a guide in career decisionmaking 

d. Formulate a personal teritative career plan which includes 
short-range and l^rig-range steps needed to carry out the 
career plan (page 122) 

<4) 1. Students identify characteristics about themselves 
which will allow them to select appropriate careers: 

-interests, activities providing enjoyment 
-abi 1 i ties, talents, aptitudes, ski 1 Is 
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-values, job satisfactions, experiences 

-freedom to make d.>*.isions and take responsibility 

-companionship, working with others 

-self-esteem, creativity 

-sel f-di recti on 

-self-approval, pride 

-etc. (page 123) 

<3) 9. Have students match their personal traits, 
interests, values, and aptitudes with job tasks, 
requirements, preparation, and benefits (page 124). 

(6) 17. Help students group job titles into workable size 
units for exploration. Emphasize the importance of 
preparing for a career cluster instead of a specific job... 
(page 126). ^ 

(7) 18. Have students practice writing a resume (page 126:). 
lQ£,I,2du2li2!l to TechriQiogy 

The first half of this program is designed to introduce 
students in grade seven to the nature of technology and its use 
m satisfying people's needs and wants. The second half, for 
students in grade eight, focuses on (1) technological concepts, 
(2) «:echnical processes, (3) methods people can use to control 
technological concepts and processes, (4) technological career 
opportunities, and (5) implication of technology to individuals 
and society. 

A major theme occurring throughout the activities of 
IHlLS^uct i.on to Technol_ogj^ in grade 7 iss 

Q§]1£££ BslstfiS! iQlSIDJf ti2i2 

Information about career "opportunities should be infused 
wherever appropriate. It should be stressed that technology 
has affected all jobs, not just those in "high-tech" areas. 
The student should be made aware at all times of 
employability skills such as dependability, honesty, 
punctuality, reliability, responsibility, ability to work 
with others, pride in work, self-awareness, sel f-rel ianre 
and self-worth (Grade 7 Syllabus, page 6). 

The use of occupational information in instructional 
strategies is also stressed throughout this syllabus. Specific 
uses include car**r «war*n*ss (defined as "a vciriety of 
experiences in which students learn about various career 
clusters, work requirements and job-readiness skills") and career 
exploration (defined as "investigating the necessary preparation, 
opportunities for jobs and potential for reward and fulfillment 
m various careers one might attempt to enter.") Decision 
making, "the process of identifying, analyzing and choosing among 
alternatives, then acting on and evaluating the results of the 
choice,") is also an activity for the course (Glossary, pages 2-3.) 

The use of occupational information for career exploration 
and decision makinj activities is also stressed in Grade 8 
Syllabus (pages 3 and 41). 



ERIC 



3 

211 



CIDS Evaluation, Appendix I, 11 



ERIC 



For example, the Introduction To lechnologij Syllabus for 
Grade 8 includes ihe following student performance objective and 
supporting competencies: 

Given an example of a new industry in each of the three areas 
of technology, students will describe how new technologies 
have created new jobs, ... then develop a training program 
to impart the broad-based skills necessary for employment in 
one of the career clusters. 

In order to do this, the student must be able to: 
a. Identify specific new Jobs in each of the three areas of 
technology and their requirements. 

b« Identify major career clusters in each of the three areas 
o'l technology. 

c. List the transferable skills common to a career cluster 
(page 43) . 

Two instructional strategies for eighth grade are: "List 
careers and necessary training for oeople involved in service 
occupations*' (page 30) and **Write a short career outline on 
medical careers which use new technologies to diagnose and tr^'-at 
medical problems'* (page 31). 

lQ&£SStU£il2Q l2 Qi£yBati2£lS 

This is a course required in ninth and tenth grades for all 
students planning ah area of concentration in an occupational 
education area. iQiLSdyctifiQ l2 Q££yE§tion§ is described as: 
"part of a continuum of Occupational Education curriculum... 
designed to include instruction in all areas of Occupational 
Education** (••Curriculum Overview", page 1). 

The two purposes of this program are: (1) *'to develop 
transferable skills which can be used in later work or home 
responsibilities" and (2) "to explore occupational interests and 
abilities prior to taking more specialized occupational courses" 
(page 1>. 

Two modules, "Working Citizen" and "Personal Resource 
Management**, are required of all students in the program. In 
addition, two other modules are required and can be chosen from 
the other twenty-two available. The general needs for 
occupational data defined in the "Curriculum Overview" will be 
discussed as well as the needs identified for the two required 
modules. 

Of the seven process skills in the **Curriculum Overview", 
two directly relate to occupational information and career 
informati^on delivery systems: 

Carger Qfit igns 

Career selection is an ongoing process addressing a 
comprehensive plan of flexible alternatives to deal with 
educational needs, occupational needs, and life career 
needs* Career options include: identification of the 
interrelationship of goalsi values clarification; evaluation 
of interests and abilities; and career investigations (page 5). 
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B££ision Making /Problem Solving 

Ioj!inn"i"'^^*^*"* ^^0P^^cIu.™of decision making/problem 
solving techniques and processes to work and persnnal 
experiences. Students will determine s'lort-term a d Inna- 
term goals, address immediate needs, and i denU f y str at2ni - . 
for successful resolution (page 6). iaeni:iTy strategies 

the rlafi??;! Vf "9 . t i sen " is "to introduce students to 

the realities of the working world" (piige 2). 

Every working citizen should be able to manage information 
asseislii'^il/r""'"" -^^^"-tly, while reafistici ly^ 
Fu?thli^?r^ P*''="='"«l s^^ills and limitations; 

InnJ^n^w ^^'^ knowledge and skill should be appli;d 
f -Snf^'i '2 ^"'^ occupational selection, • 

acquisition, and maintenance (page 2> . 

The "Working Citizen" module contains five ma ir.r = 

p:rso;%'i"Sivei"'°'"r --3%j^rn%:r^iat^on':;-,d' 

personal development as it applies to the world of w.-.rk (Daae '^•) 
The ive topics in the module are "designed to provi de stSdent^ 
with information and experience which will help them Lke future 
choices concerning work" (page 2). future 

Ten of the nineteen skill areas the student is exnect^ri 
develop during this module relate to the use of oo upijfona? 
information. They are: ■-'t .upational 

fndiCr;!If P^'-sc.nal skills and talents, relating them to 
individual choice of employment 

6. Develop personal goals for education, training, and 
future employment «»*riing, ana 

7. List and explain personal characteristics and Qualities 
that are desirable for successful employment ""^^^t^^s 

8. Describe and utilize sources of employment information 
S. Determine the relationship of education and trainrng to 
acquisition an.- maintenance of employment ^"-aming to 
10. Identify ot least two positions within two specific 

the^sfxJf"^' ''^'''l interest to the student, anS describe 
the skills required for each position uescrioe 

tJ^infnS*'^^ occupational positions that require advanced 

14. Analyze the working conditions of a chosen emolovment 
area, and explain the influence of working conditions on 
?^^^i''f^^"'^ health, as well as safety '"'"'"'"^ 

16. Match personal job competencies with specific 
employment opportunities 

17. Analyze the factors that influence individual job 
selection (pages 2-3). 

In the outline for this module, five of the ma ior units 
require occupational and career information (pages 4-6) 5n 
addition, its use is required to meet 'per f or manj^ object i^es/ 
supporting competencies' listed throughout the draft syUabus 
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Exc#rpts from "District Suidance Plan: 



Imp lamenting Commissioner's Regulation 100.2 (J)'* 



In November l^iB4, by action of the Board of Regents of the 
State of New York^ a new Part 100 of the Commissioner's 
Regulations was adopted which r?)odified the original requirements 
for district guidance plans. "The Regulation requires each school 
district to develop a district guidance plan to serve students 
kindergarten through twelfth grade (page 1>. 

The Regulations state in Section 100.2 (J): 

(1) Public Schools (and non-public schools). Each school 
district shall have a guidance program for all students... 
(ii) In grades 7-12^ the guida.ice program shall include 



the following activities or services: 

(a) an annual review of each student's educational 
programs and career plans, with such reviews to be 
conducted with each student individually or with small 
groups by personnel certified or licensed as school 
counselors; 

(b) instruction at each grade level to help students 
learn about various careers and about career planning 
skills conducted by personnel certified or licensed as 
school counselors, or by classroom teachers in 
cooperation with school counselors; 

(c > other advi spry and i ndi vi dual or group counsel i ng 
assistance to enable students to benefit from the 
curriculum, to help students develop and implement 
postsecondary education and career plans,... 



The State Education Department issued guidelines to asrist 
district personnel to develop "guidance plans which meet the 
needs of their students consistent with Commissioner's Regulation 
100.2 (J)." ("District Guidance Plan, page 1). In the Glossary, 
"Careers and ... career planning skills" are defined as the 
knowledge, understandings, and abilities related to the major 
guidance themes of self awareness/understanding, decision-making, 
planning, information gathering, values clarification and 
employability (page 4). 

The document further states that: 

"Guidance program for all students" includes the planned 
program activities and services of personnel to enable 
students to benefit from the educational programt. to develop 
the knowledge, skills and abilities necessary to implement 
career plans, to assist in their personal and social 
development, and to develop problem solving skills and 
techniques (page 5>; Cand that!] "Instruction at each grade 
level to help students learn about various careers and about 
career planning skills conducted by personnel certified or 
licensed as school counselors, or by classroom teachers in 
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cooperation with school counselors" requires that counselors 
or classroom teachers must provide planned learning 
experiences tc develop the career planning skills of 
students. This permits career education programs and 
activities to be delivered by other than counselors. It 
requires cooperation with counselors to assure a coordinated 
approach se that counselors, in their individual work with 
students^ will reinforce the knowledge and skills gained in 
instruction in the personal career planning by the student. 
These experiences may be provided to groups of classroom 
size or smaller (pages 5-6). 

The Guide offers suggestions for Section E, Activities to 
take place during grades 7-12 which might accompany an objective, 
••Instruction in careers and career planning." These activities 
include: 

Sel f -evaluation (grade 7); 

Use of career reference materials in library and Career 

Resource Center (gr^de 7); 
Identification of Job clusters related to student's own 

interests and abilities (grade 8) 
Develop list of appropriate courses to be studied during the 

next school year — keeping in mind tentative plans for 

«:ollege major and/or career goals (grade 9; 
Investigating education and training requirements for 

occupations identified in Self-Directed Search Survey form 

(grade 9); 

Orientation session by counselor re: Guidance Awareness 
(grade 9> ; 

Values Clarification exercise (grade 10); 
Decision-making exercise (grade 10) j 
Self-awareness activities (grade 10); 
Career -awareness activities (grade 10); 

Group guidance sessions at which students are instructed re: 
factors important in choosing a college (grade 11); and 

Group guidance sessions re: preparing and updating a resume 
for college and/or potential employers (grade 12). (pages 
27-31). 

Since Guidance Plans must be reviewed each school year, the 
Guide offers suggestions for "student outcomes" which could serve 
as the basis for the review. Some of the suggested outcomes 
which are related to career planning and decision making are: 

7th Graders will: 

o learn more about themselves by identifying their 

interests, abilities, aptitudes, values, and personal 

characteristics, and 
o be aware of their interests and of occupations related 

to their interests as derived form completion of a 

vocational interest inventory. 
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8th Graders wil 1 : 

o begin preparation of th<.ir personal educational plan; 
o understand that different jobs require different 

types of education and/or training; 
o understand the basic relationships between school 

subjects and eventual occupational requirements; and 
o identify related occupations for each subject they 

are studyi ng. 

9th Graders wi 1 1 : 

o understand that gathering information is an important 

step in educational and career planning, and 
o identify and list various types of information required. 

10th Graders will : 

o recognize a number of ways in which occupations may 
be grouped. 

11th Graders will : 

o set a long-range educational goal, and 

o develop a written action plan for reaching the goal. 

12th Graders will: 

o demonstrate effective job interview skills in a role- 
play situation (pages 43^44). 

The review guide which will be used by State Education 
Department reviewers in the formal review to determine compliance 
of district plans includes the following career guidance outcomes 
for grades 7-12. 

9. An ANNUAL REVIEW of each student's educational progress 
and career plans is conducted with each student individually 
or with small groups of students by personnel certified or 
licensed as school counselors. 

10. INSTRUCTION AT EACH GRADE LEVEL to help students learn 
about various careers and about career planning skills is 
conducted by personnel certified or licensed as school 
counselors, or be classroom teachers in cooperation with 
school counselors. 

12. Other ADVISORY ASSISTANCE to help students develop and 
implement post secondary education and CAREER PLANS is 
provided by teachers or counselors, or be certified teaching 
assistants under the supervision of cour . elors or teachers. 

17. Other individual or group COUNSELING to help students 
develop and implement post secondary education and 'CAREER 
PLANS is provided by certified or licensed school 
counselors, or by certified or licensed school 
psychologists, or by certified or licenses school social 
workers in cooperation with school counselors (pages 4S- 
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Appendix II 
SURVEYS OF USER SATISFACTION: 
INSTRUMENTS AND RELATED MATERIALS 

A. Principal's Survey of User Satisfaction 

B. Letter from Deputy Commissioner, James A. Kadamus, New York 
State Education Department 

C. Letter to Principals 

D. Special Notes for Principals 

E. Counselor's Survey of User Satisfaction 

F. Letter to Counselors 

G. Student's Survey of User Satisfaction 

H. Letter to Students 

I. Parent 's Survey of User Satisfaction 
J. Letter to Parents 
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PRINCIPAL'S SURVEY OF USER SATISFACTION 
PRINCIPAL'S NAME,^ 
FULL NAME OF SCHOOL 

CITY, TOWN, OR COUNTY IN WHICH SCHOOL IS LOCATED 

NAME OF COMPUTERIZED CAREER INFORMATION DELIVERY SYSTEM USED IN 

YOUR SCHOOL: ^ check one) CHOICES Discover GIS 

MetroGuide SIGI Other: 

We would like do acknowledge your participation in the final 
report of this study. May we do so? 

YES NO 



BIEiQIIQIilS: Please circle your response to each item on this 
survey. If you wish to make acditional corriment s, please feel 
free to do so. 



A. Computerized career information delivery systems (CIDS) are 

used in different setting^^ and with different groups within a 
school. Circle YES or NO to indicate whether or not your school 
has used the CIDS to support each activity listed below. 

1. Individual career counseling YES NO 

2. Career education/awareness groups or classes YES NO 

3. College guidance YES NO 

4. Special education programs YES NO 

5. Occupational education programs YES NO 
S. Basic skills or remedial reading YES NO 

7. Subject class instruction such as English, 

economics, etc. YES NO 

8. Job placement YES NO 

9. Employabi 1 i ty skills groups or classes YES NO 

10. Counseling physically handicapped students YES NO 
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PRINCIPAL'S SURVEY: NAME OF SCHOOL 



B. For €?ach of the following items, please indicate your level of 
agreement or disagreement with the statement by circling the 
appropriate number. 



14 T.. ^^.1 I ^ ^- ^ ' J Strongly Ftithtr Arrtt Stronclv 

1. In general, the school is satisfied Ar»t Acrtt NorDiMgitt DtMcrtt i>«*jr/t 

wi th your computer i zed career 

information delivery system (CIDS). 5 4 3 2 i 

12. Students have expressed the usefulness 
of the CIDS in their career or 

educational planning. S 4 3 2 1 ' 

13. The information presented in the 
CIDS is appropriate for the students 

in my school. 5 4 3 2 1 

14. Parents have expressed their 
satisfaction with he information 

their children get from the CIDS. 5 4 3 2 1 

15. The system has been used successfully 
for instruction in subject areas, as 

well as for guidance purposes. 5 4 3 2 1 

16. The system has been accepted easily 

by counselors. 5 4 3 ' 2 1 

17. The system is well worth the cost. 5 4 3 2 1 

18. The CIDS enables counselors to make 

better use of their time. 5 4 3 2 1 

19. The amount of time it took to train 

counselors was appropriate. 5 4 3 2 1 



20. There have been no significant 

problems with scheduling students 
to use the system. 



PLEASE RETURN COMPLETED SURVEY BEFORE APRIL 30, 1986 TO; 

Dr. Deborah P. Bloch 
Post Office Box. 20369 
Cherokee Station Post Office, 
New York, New York 10028-9991 
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TD fixn or LEAKMNG 

THE STATE EDUCATION DEPARTMENT/ THE UNIVEPSl'^Y OF THE STATE OF NEW YOPK/ ALBANY N Y 12234 

ASSiSTANTCOMMtSSlONER FOR OCCU PATIONAL AN D CONTi NU t NG EDUCATION 
THE NEW YORK STATE EDUCATION DEPARTMENT 
ALBANY. NEW YORK 12234 

OCE # 13 



March 10, 1986 



TO I 



FROM 



SUBJECT: 



District Superintendents of Schools 
Superintendents of Selected School Districts 

James A. Kadamus , Assistant Commission!^ | 
and Chairman, NYSOICC ^fCtMs^^ Ct * fCjfJSf^'O^^'^'"'^ 

Announcement of Special ^udy for Computerized Career 
Information Systems 



A new study of computerized career information systems was 
recently initiated by the State of New York, Through a compet- 
itive proposal process the Education Department and the Depart- 
ment of Labor, in cooperation with, the New York State Occupa- 
tional Information Coordinating Committee have selected Deborah 
Perlmutter Bloch and Joyce Ford Kinnison as the contractors to 
conduct the study. 



In the course of the study. Dr. Bloch and Ms 
need to gather information from the following: 



Kinnison will 



School districts and BOCES 

offices of the State Education Department including the 
Office of Vocational Rehabilitation 
- Offices of the Department of Labor 

Office of the Commission for the Blind and Visually 
Impaired 

To gather the needed information, surveys of a sample of 
administrators, clients, students and parents will be conducted 
and a sample of sites utilizing career information delivery 
systems will be visited. 
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This study is of great importance to the State of New York 
since it will be an initial step to insure the appropriateness 
and accuracy of the delivery of career information in the 
future. Wer thereforer urge your cooperation in responding to 
requests from Dx, Bloch and Ms. Kinnison and offer you our 
assurance that all information on individuals and indivi.f.ual 
agencies will be kept confidential and presented only as aggre- 
gate group data. 

If you are contacted for a survey or site visit and would 
like additional information, please contact: 



Barbara Shay, Chief 

Bureau of Oc cupational Education Po licy 

Development 
New York State Education Department 
One Commerce Plaza, Room 1624 
Albany, NY 12234 
(518) 473-7408 



cc: Directors of Occupatio-nal Education 
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Deborah Fbrlmutter Bloch, Ph.D 



April 7, 1986 
Dear Principal : 

The State of New York Is conducting a study to evaluate the 
current use of computerized career information delivery systems 
in order to plan for their future use. Your school has been 
selected to participate In a survey to determine the level of 
satisfaction vith systems currently in use. 

The attached letter from James A. Kadamus, Deputy 
Commissioner for Occupational and Continuing Education at the 
State Education Departments explains the study and requests your 
participation and cooperation. 

Participation in this study involves only two activities on 
your part. The first is to complete the enclosed Principal's 
survey of user satisfaction and return it before April 30^ 1986. 
The second Is to distribute surveys of user satisfaction to a 
counselor or teacher and to ten students. Full instructions are 
on the attached sheet. 

If you have any questions, please call Dr. Deborah 
Perlmutter Bloch at <212> 794-1098. 

We look forward to receiving your information so that the 
final report and recommendations will accurately reflect your 
school's experience with a computerized career infov^m^tion 
system. 

Sincerely yours. 



Deborah P. Bloch 



Enclosures 
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SPECIAL NOTES FOR PRINCIPALS 



SURVEY OF USER SATISFACTION 
WITH COMPUTERIZED CAREER INFORMATION DELIVERY SYSTEMS 



1. Please complete the attached Principal's Survey of User 
Satisfaction and return It before April 30, 1986. 

2. Ask the professional person In your school vho Is most 
familiar with the computerized career Information delivery system 
to complete the Counselor's Survey of User Satisfaction and 
return It before April 30, 1986. 

3. Select, or have the appropriate professional select, ten 
students who have used your computerized career information 
delivery system and ask each of the students to complete and 
return the Student's Survey of User Satisfaction. 

4. Ask the same ten students who have used your computerized 
career Information delivery system to have their parents complete 
and return the attached Parent's Survey of User Satisfaction. 



NOTE: You are NOT responsible for collecting or returning any of 
the completed surveys except your own. Envelopes have been 
provided for each respondent. We hope this effort to keep your 
administrative responsibility to a minimum encourages your 
participation in this study. 

NOTE: At no time will you or your staff be asked for the names 
of, or any other Information about, the student or parent 
respondents . 

NOTE: The information In all reports of the study will be 
aggregate data. No Identification of the responses by 
Individuals or schools will be possible. 



Thank you very much for your cooperation. 
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COUNSELOR'S SURVEY OF USER SATISFACTION 

• 

FULL NAME OF SCHOOL 

CITY, TOWN OR COUNTY WHERE SCHOOL IS LOCATED 

NA\iE or THE COMPUTERIZED CAREER INFORMATION DELIVERY SYSTEM YOU USE: 

CHOICES Discover GIS ^.M^troGui d# SIGI 

Oth#rx 

POSITION OF RESPONDENT CP1«M« ch*ck on*)t 

Coun»»l or 

, , „Soc i a 1 Wor k •r 
.„_T#«ch*r 
__L i br ar i an 

,_^"oth*r: Pl#*s* fill in ^ 

A. Computerized care^^r information delivery systeirtS (CIDS) are 
used in different settings and with different groups within a 
school. Circle YES or NO to indicate whether or not you have used 
the CIC3 in your school to support each activity listed belcw. 

1. Individual career counseling YES NO 

2. Career education/awareness groups or class<>s YES NO 

3. College guidance YES NO 

4. Special education programs YES NO 

5. Occupational educat ion programs YES NO 

6. Basic skills or remedial reading YES NO 

7. Subject class instruction sucn as Enr^Iish, 

economics, etc. YES NO 

8. Job placement YES ND 

9. Ehipl oyabi 1 i ty skills groups or classes YES NO 

10. Counseling physically handicapped students YES NO 

B» Computerized career information delivery systems (CIDS) and 
their printed materials are used to help students at various 
po;^nts in the career decision making process* Considering both 
group and individual use, indicate whether or not you have 
observed each of the following outcomes for users of the CIDS in 
your school. 

12« Increased awareness of self in relation to 

occupational or educational choices YES NO 

12. increased awareness of the transferability of 

skills acquired in one occupation to those skills 

needed in other occupations YES NO 

as. Gained gene-al knowledge of the labor market YES NO 

14. Gained gen^^ral knowledge about careers VE5 NO 

15. Gained information about Job seeking skills YES NO 

16. Gained general infc*mation about educ^ti^jKal 

and training options YES NO 

17. Gained information about specific ^educational 
and training institutions, such as colleg'>Sr 
trade schools, hiQh schools, and/or short-term 

training programs YES NO 

18. Gained information about financial aid 

opportunities YES NO 

19. Made a decision about a career choice YES MO 
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COUNSELOR'S SURVEY. NAME OF SCHOOL 



20. Mad* a decision about further education or 

training YES NO 



C. List^bd b«Iow ar* i n f oritiat i on components and related hiaterials 
found in many computerized career information delivery systems. 
Please indicate the degree of usefulness of each item as it is 
presented in the CIDS you are using in your school. 



Sit^»»ly V»rf MM*r«Uly Hel Vtry Not At Ail Hot 
UMfuI Uvctul \Huivl ^tu] Uwtu] AppLcAbI* 



21. Occupational descriptions 

22. Occupational groups or cluster 

23 . Emp I oymen t out I ook i n f or mat i on 

24. Uag<t and salary information 

25. Physical demands and activities 
required of a Job 

26. Licensing, cert i f i cat i on, and 
teste required for a Job 

27 . In f 6r mat i on about r el at ed 
occupations 

28. Information about educational 
or training programs related 
to occupations 

29. Descriptions of secondary and/ 
or post-secondary educational 
and training programs 

30. Information about occupations 
related to educatiQn and 
train! ng programs 

31* Educational and training 
institution descriptions 

32. Tinanciai aid and scholarship 
information 

33. Employability skills information 
(resume wr i t ing ^ int er vi ewi ng, 
etc. > 

34. User (student) handbooks and 
guides 

35. Interactive occupational 
exploration strategy 

36. Ability for users to query system 
as to iihy a specific occupations 
did. not appear on their lists 

37. Interactive educational insti* 
tutions exploration strategy 

38. Methods of direct access to 

oc c upat i onal in for mat i on f i 1 es 

3So Methods of direct access to 
educational and training 
i n f or mat i on f i 1 es 

40* Links to published assessment 
instruments 



5 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 10 

5 4 3 2 1 0 

5 4 3 2 1 0 

S 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 10 

5 4 3 2 1 ^ 

5 4 3 2 1 ^ 

S 4 3 2 1 0 

5 4 2 2 10 

5 4 3 2 10 

5 4 3 2 10 

5 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 1 0 

5 4 3 2 1 0 
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COUNSELOR'S SURVEYS NAflE OF SCHOOL. 



D. Tr«ining for counselors «nd other professionAls which focuses on 
the use of « computerised career information delivery system (CIDS), 
its hardware, its inforrnation files, and its user materials is 
considered to be an important factor in the usefulness of a CIDS in 
a school* Please answer the following questions abou^ the training 
you received on the system you are using in your school. 

41* Did you receive training in the use of your 

CIDS? YES NO 

(Tf NO, skip to section E.) 

42. If YES, from whom? 

„.„vendor or central distr ibutor 

someone in your school or school district 

^„„other : please ^p^c i f y„„„„«„„„„«^„„„„«„...^. 



43. How uB^fu^ was it? S 4 3 2 1 

£• Counselor manuals are usually part of a total CIDS packagi^. 
Please answ^er the following questions about the counselor materials 
which are part of the system your szhool is using. 

44* Do you have a counselor's handbook or 

implementation guid«? YES NO 

(If NO, skip to section F. > 

4S. How useful is it? 5 4 3 2 1 



r. The next two questions d^al with the lergth of time it 
ordinariliy takes a student to go through a session with the CIDS. 

4£. Uhat is the average tin'ie per session for systerii u:sers? 

lees than 30 niinules 



^„_^30 minutes to 1 hour 

^..„ftior* than 1 hour 

47. In your opinion, what is the iJeal length of £ime it should 
take a student to complete a session a^ the computer ?^^„^ 



6» It is important that CIDS software be designed so that the 
student does not have problems using the program. In addition, 
the information which is presented in the CIDS should be easy to 
understand* Please answer the following questions about the "user 
friendliness** of the software aiid the presentation of information. 

48. The CIDS program which my school is using is 

easy for most of the students to use. YES NO 

4S. The information which students receive while 

using the CIDS is easy for them to understand. YES NO 



SO. Xf you were designing your ideal computerized career 
information delivery system, what would be its most important 
features? (Please use the back of this sheet for your answer^) 



PLEASE RETURN COMPLETED SURVEY BEFORE April 30, 1986 TOi 

Dr. Deborah P, Bloch 
Post Office Box 20363 
Cherov.ee Station Post Office 
New York, New York 10028-9991 
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Deborah RcrlmutterBloch^PhD 



April 7, 1986 



Dear Counselor: 



The State of New York is conducting a study to evaluate the 
current use of computerized career information delivery systems 
in order to plan for their future use. 

Your school has been selected to participate in a survey to 
determine the areas of satisfaction with computerized career 
Information delivery systems now in use. The attached letter 
from James A. Kadamus, Assistant Commissioner for Occupational 
and Continuing Education, explains tLo study. 

As a counselor or other professional using a career 
information system, you can provide essential information about 
the current use of computerized systems and their efficacy* We, 
therefore, appreciate your willingness to complete the attached 
survey • 

Some of the questions on the survey are to be answered by 
circling YES or NO to reflect your answer. For others, you 
iilmply circle a numerical rating* The final question provides 
you the opportunity for an extended reply* 

Please fill in the information requested at the top of the 
survey form and on each page of the survey, complete the survey, 
and return it in the attached envelope before April 30, 1986* 

Thank you for your cooperation in this important matter. 



Sincerely, 



Deborah P. Bloch 




STUDENT'S SURVEY DF USER SATISFACTION 
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FULL NAME DF THE SCHOOL YDU ATTEND. 



CITY, TDWN, DR CDUNTY IN WHICH YDUR SCHDDL IS LOCATED 

YOUR GRADE IN SCHOOL (Circl* on*> 9 10 11 

NAME OF THE COMPUTERIZED SYSTEM YOU USED: (ch*ck one) 

CHOICES Discover GIS ^^^Mfrtrcgui 

SIGI Oth^r 

HOW MANY TIMES HAVE YOU USED THE SYSTEM? 



1. B«for« you us«d th«' system, w«r# you given « 
user handbook or a guide to help you use the 

system? YES NO 

Clf NOt skip to question 3.? 

2. If YES| was the user handbook or guide useful 

in helping you to use and understr^nd the systeru? YES NO 

3. Was the system easy to use? YES NO 

4. Was the inforiration preBent«>d by the system 

easy to read and understand? YES NO 

S» Did the computerised system help you learn 

something new about occupations? YES NO 

&• Did the computerized system help you learn 
more than you knew b-irfore about how to make 

a decision about a career? YES NO 

7. Did the computerized system help you 
understand something about tK« relationship 

between occupations and your interests? YES NO 

8. Did the computerized system help you 
understand something about the relationship 

between occupations and your abilities? YES NO 

3. Did the computerized system help you learn 
something about the relationship between 

occupations and education or training progr«inS? YES NO 

10. Did the computerized system help you learn 
something about how to prepare for an 

occupation? YES NO 

11. Did using the computerized systesi help you 

makd a decision about a career? YES NO 

12. Did you get a printout to take away with you 
which contained the information you received 

while using the system? YES NO 

12. Was it uFeful to you to be able to keep the 

printout for future reference? YES NO 

14. Did you like using a computerized t^ystem to _ 

get career information? YES NO 

15. Would you recommend the system to someone who 

needs the same kind of information you need? YE£: NO 

If you have any suggestions to help make a computerized system 
more useful, please write them on the back of this sheet. 

PLEASE RETURN YOU'^. COMPLETED SURVEY IN THE ENCLOSED ENVELOPE BEFORE 
APRIL 30, 1986 TOx 

Dr. Deborah P. Bloch 
Post Office Do>: 20369 
Cherokee Station Post Office 
New York, New York 1002B-S931 
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Deborah ftrrlmutter Bloch, PhD 



April 1, 1986 



Dear Student: 



The State of New York Is carrying out a study of career 

Information delivery systems* The purpose of the study to see how 

well the computerized systems that are now used are meeting 
students ' needs for career Inf ormat 1 on , 

As a person who has used a computerized system# you can 
provide Information essential to this study. The Information 

will be used to Improve systems for your use In the future and 
for the future use of other people. 

The attached survey Is made up df fifteen questions. Please 
fill In the Information asked for at the top of the survey form 
and answer each question by circling YES or NO. 

After yow have completed the survey, return It In the 
addressed, stamped envelope which Is Included with this letter 
before April 30, 1986. 

Thank you for your cooperation. 



Sincerely yours. 



Deborah P. Bloch 
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PARENTIS SURVEY OF USER SATISFACTION 
NAME OF THE HIGH SCHOOL YOUR CHILD ATTENDS:^. , 



CITY, TOWN, OR COUNTY IN WHICH THE SCHOOL IS LOCATED: 

CURRENT GRADE OF CHILD: CClrcla one) C 10 U 12 

1* Do 70U think that your child's high school 
provides the Information needed for making 
career and educational choices? YES FO 

2- A. To the best of your knowledge, has your 

child ever brought home a printout of career 
•or college Information obtained from a 

computerized system at school? YES NO 

(If NO, skip to Question 3«A*> 

B. If YES, did you find the Information valuable 

for the decisions your child is or was miking? YES NO 

3- -A. Have you ever seen your child use a computer 

in school to get career or college Information? YES NO 
(If NO, £kip to Question 4.) 

B. If YES, did you think the computerized system 

was a valuable tool? YES NO 



Computerized career Information delivery systems can be used to 
help students explore and obtain Information about occupations, 
educational and training programs, and colleges. For each of the 
following, please circle YES if you believe a computer would be a 
good tool for the purpose listed, or circle NO If you believe a 
computer would ao£ be a good tool for the purpose listed. 

4. Identify a list of careers based on a 

student's interests and abilities. YES NO 

5. Identify a list of colleges which meet a 

student's needs and interests. YES NO 

6. Clarify a student's values. YES NO 

7. Show relationships between work ;^nd 

educational programs. YES NO 

8. Deliver information about occupations. YES NO 

9. Deliver information about areas of 

ccnceatratlon in high school. YES NO 

10. Deliver information about educational 

programs In colleges. ygs NO 

11. Deliver information about colleges and 

un I vers i t ies • YES NO 

12. Deliver information about private trade 

schools t YES NO 

13. Deliver information about financial aid and 

soholarsh 1 ps • YES NO 



Thank you for your cooperation. Please return your completed 
survey In the enclosed envelope before April 30, 1986 to: 

Dr. Deborah P. Bloch 
Post Office Box 20369 
Cherokee Station Post Office 
New York, New York 10028-9991 
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April 7, 1986 
Dear Parent: 

The State of New York Is conducting a study of the career 
and educational planning needs of junior and senior high school 
students and others to see how computerized systems can be used 
to meet these needs. A letter verifying this from the Assistant 
Commissioner for Occupational and Continuing Education at the 
State Department of Education Is attached. 

As a parent of a high school student, you can provide 
Information essential to this study. We, therefore, appreciate 
your willingness to complete the attached survey. 

The survey Is made up of fourteen Items. Please fill In the 
heading Information, answer each question by circling either 
•Yes" or •No", and return the survey before April 30, 1986. A 
stamped, addressed envelope is enclosed for your convenience. 

Thank you for your cooperation. 

Sincere 1 y. 



Deborah P. Bloch 



Enclosures 
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PARTICIPATING PRINCIPALS 



We would like to thank the following sch ol principals and 
to acknowledge their participation in. cooperation with, and 
support of this study. Not only did they complete their 
Principal's Survey of User Satisfaction, they also facilitated 
the distribution of the other survery to the appropriate groups 
m their schools. Their interest and efforts contributed greatly 
to insuring the return of surveys and to making this study a 
success. 



Morton Dameseck. Principal 

Franklin K. Lane High School 
Brooklyn. NY 
Steven C. Applebaum, Principal 

Prospect Heights High School 
Brooklyn. NY 
Saul F. Baily. Principal 

High School of Fashion Industries 
New York. NY 
Joan Beutler. Principal 

Port Chester High School 
Port Chester. NY 
Robert P. Burns. Principal 

Francis Lewis High School 
Fresh Meadows, NY 
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R. David Carhart, Principal 

Liverpool High School 

Liverpool, NY 
John J. Dinneen, Principal 

Madrid-Wadding ton Central High School 

Madrid, NY 
Barry M. Gould, Principal 

Cooperstown Junior-Senior High School 

Coopers town, NY 
Jack L. Groveman, Principal 

Forest Hills High School 

Forest Hills, NY 
Edward Horn, Principal 

Adlai E. Stevenson High School 

Bronx, NY 
Richard A. Klein, Principal 

LaGuardia High School of the Arts 

New York, NY 
Frederick J. Koury, Principal 

City-As School 

New York, NY 
Robert Leder, Principal 

Herbert H. Lehman High School 

Bronx, NY 
Allen G. Leibowitz, Principal 
New Utrecht High School 
Brooklyn, NY 
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Bertram L. Linder, Principal 

Benjamin N. Cardozo High School 
Bayside, NY 
Michael J. Lobasso 

Babylon Junior-Senior High School 
Babylon, NY 
Betty Anne McDonough, Principal 

George Washington Vocational and Technical High School 
Brooklyn, NY 
Edwin J. Martin, Jr., Principal 

New Hartford Senior High School 
New Hartford, NY 
H. D. Mettelman, Principal 
Camden High School 
Camden, NY 
Ralph S. Musco, Principal 

Susan E. Wagner High School 
Staten Island, NY 
John Olsen, Principal 

Williamsville East High School 
Amherst, NY 
Thomas J. Readyoff, Principal 
Mahopac High School 
Mahopac, NY 
Paul Ripchik, Principal 

Saratoga Springs High School 
Saratoga Springs, NY 
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Randolph H. Ross, Principal 
Newtown High School 
Queens, NY 

Paul Skinner, Principal 

Richfield Springs Central 
Richfield Springs, NY 

Glenn Young, Principal 
Batavia High School 
Batavia, NY 
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J. A>^.Ci_ Cdn7Q<>ok. - bs>^. C/hs .ca^ jLd^ .^n^j^ 



IMJL ■ C //ucIo(Je. 'hai^u^^ fir€z£ia^ j2u^}$ci^<^^ 

ft. dfren^^ 
/ 
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REQUEST FOR INFORMATION 
FOR THE 

...^T«I!^^^ ^^^^^ EDUCATION DEPARTMENT 
"EVALUATION OF COMPUTERIZED CAREER INFORMATION SYSTEMS- 



NAME OF SYSTEM 

ADDRESS 

^^"•"Y STATE ZIP 



TELEPHONE 



BiEieiiQNS 

I*^*® request for information <RFI) is for the New York State 
Education Department "Evaluation of Computerized Career 
Information Systems." This study and other activities associated 
with It were described to you in a letter from Dr. Deborah 
Perlmutter Bloch dated May 14, 1986. 

The ovet-all purpose of the study is to identify the 
computerized career information delivery system or systems 
which can serve a broad range of uses including: 

1. Impleraentf-tion of three new occupational education 
curricula for the 7th, 8th and 9th grades — Home and Career 
Skills, Introduction to Technology and Introduction to 
Occupations; 

2. Implementation of guidance plans in the junior and senior 
nigh schools! and 

3. Assisting career and educational planning of individuals 
m such groups as Junior and senior high school students, 
out-of-Bchool youth, community and technical college 
students, handicapped workers and other adults. 

The RFI consists of ten major questions, most of which have 
many parts. Questions I through V require checks or brief 
responses. Please answer them directly on this form. Since 
"* ^" through X require extended answers, please use 
additional sheets of paper, numbering them to correspond to the 
questions. Question VI requires that you provide printouts of 
mformtion from your system. Please attach the printouts 
requested to this form. 

^^T« SLS'^'^ "^^^^ SYSTEM NAME AT THE TOP OF EACH PAGE OF 

piS^r SLS^li SEPARATE RESPONSE SHEETS AND TO SIGN 

THE FINAL PAGE OF THIS DOCUMENT. 

Tft 11*^:^ completed form POSTMARKED NO LATER THAN JUNE 16,1986 

TOs Deborah P. Bloch 

Post Office Box 20369 

Cherokee Station Post Office 

New York, NY 10028-9991. 
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System Name 2 

I. MAJOR FILES OR COMPONENTS 

The following items have been identified as the major 
component elements of a computer-based career information system 
For each component, please check YES if it is a separate file in 
yoxr system. Check NO if it is not a separate file. 
CPlease notes specific information items related to each 
component are listed in the next section.) 

QSSSSff YES NO cmms 

ky OccupatioAil IflforMtion _ 

B. Labor Itarktt Entry InforMtion 

C. SKOfidiry Prograas of Study ^ 

D. Pott-SKondiry Progrm of Study 

E. Tifo-year Colltgef 
F« roiir-y«ir Colltf ts 

8. Pofit-ffcofldiry Vocational Training Institutions 

H. Financial Aid _ 

I. Scholarships (other than governitnt sponsored or college-bas«d)_ _ 
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System Name 



II. SPECIFIC INFORMATION 

l^^.^f^^ component above, items have been identified as 
essential to a tiystem. Some of these should be available at the 
inrVT* information analysis, some should be avatJabie 

;nd*s:a?e1:ver: """" -''-tate^ 1-el, others at both national" 

columnTth*:t^'ap^;ir:o'J::r rjijeml^^^^-' ''^^^^ ^'^^^'^ 

Checking the first column, NATIONAL, indicates that your 
system currently carries information on this item 
representing a national level of data analysis. 

Checking the second column, NYS, indicates that your system 
thiS'Itim. information specific to New York Statlfor 

Checking the third column, ABLE, indicates that your 

for ?!;iri^;i;''f';*'i^ *° ""r^^ state-specific information 

Tor this Item if it were supplied to you. 

HATIOIAL ^''^'^ Cyopj Q[ AMIS 

1. OccuMtioMl titlf with idfniiryin) codf 

2. Occupational dfKriptiM 

3. Occupatioaal group or clwUr 

4. Job dutiff 

5. Inttrff»« (of Horkfrt as validattd in various studies) 

6. Aptitudes 

7. TeiperaMiits 

8. Entry level skills required for job 

9. Pfcyii<*l deiands (awwjit of wijbt the worker nust lift or carry) 

10. Physical activities 

11. Tools, equipient, etc. used on the job 

12. Enviromental/tfork conditions 

13. Hours of vork and travel required on Ihe job 

^ 14. Industries in <rfiich occupations occur (places of eaployient) 

ERIC 



System Name 



CIDS Evaluation, Appendix V, 5 
4 



!!*!I0!!«: !!!§ able 

15. Hiring cbMntlf 

16. Carttr laddtri and advaacmnt opportunitits 

17. Occupitional oatlook 

18.. CurrMt and futurt xcupational supply and dttand 

19. Current •■ployatflt (nuabtrti by occupation by industry) 

20. Earnings and bentfitf (btginningi avtragtf rang*, 

fringt btntfitf) 

21. Education/training r«quir»d 

22. Licming, ctrtificationt tnts^ ttc. required 

23. HatpfiU sKOfldary Kbool subjects _ 

24. Sflattd occupations 

23. Rflatfd iilitary occupations 
28. Sources of additional inforMtion 

27. Bias-frH data but - occupations rtprtstnt th* eotirt range of 
tbf labor narkft 

28. liu-frtf occupational titlM and contents _ 

2). Occapational classification and coding systto based on the DOT or 
or SOC 

!i l&iSLHiiss csttoofaii itlild is i^ot mii Ma 

30. Job seeking skills 

31. Resuie vriting 

32. Interview techniques 

33. Eaployaent services ^ 

34. Self-eoploytent and entrepreneurial inforoation 
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System Name [ 

^ IfllSHitigg CfittSSSfii! kiUifd to SKsndarx Ams of tonctntNti oa 

MJJONAL NVS ABLE 

ArM of cofiCMtritioA dtKriptioA 

36. Arti of coocffltritioii group or duftor 

37. Typical covrso vork 

38. Typtf of Ifirning activitios 

39. Skills Md kaovltdgt ic^uirni 

40. RflitMt entry 1ml KCiip«tions 

41. RfUttd occupitioflf requiring further study 

42. RfUted post-SKondary progrm 

h lalsmtltt csmmh KtUtfS to PsjtiSsiaaiaa liucatioHi ind Traiaiia '[saLiis 

43. Prograi dticriptioii 

44. Prograt gro^p or dufttr 

45. Typical courtt vork 

46. TypH of Ifiriing ictivitifs ^ 

47. Skills Md knovlidgt icqoirtd 

48. CtrtificatH or dtgrm urntd 

49. MorMl Itngth of tiat to co^iUtt CKtificatt or degroo _ 

49. RfUtfd occupatioof 

50. Rflatid progratf of study 

51. Typf(s) of institutions offering prograa 
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£s iSlSrulisn CMgontiitf R»lit«d to Educational and Jrjinigg Institutions 

N.B.: For this SKtion, you Mst chKk MTIOML, IIY8, « ABtE 2 tius for Nch itn. 

iifor m m:m mim mmm trainikg 

mST'lB. (PWT-Sic.) 

mim m m mim "^""m 

52. NsM and addrtss of educational or training institution _ 

53. SMgraphic Ixatioit 

54. Typ» of instittttioa 

55. MiissioR RtquirnMts (ttst scorn, application proc*dar»s, 

application dcadlin*, fMs, physical Mai, ttc.) _ 

56. Nait and address of adnissions contact person 

57. Cost (in-^tate and o«t-of state tuition, root, board, fees) 

58. Housing (availability, on-ciapus, off-canpos, percent of 

undergraduat-s living on ca^ms, refuircd of fresbten, etc.) _ 

59. Financial aid inf«rMtion (soirees, application prxtdures, 

application deadline, average award, etc.) 

(0. Naie and address of financial aid contact person 
fil. Education and training prograas offered 

62. Types of cKtificates or degrees awarded 

63. NofMl length of tiie to coiplete certificates/degrees offered 

64. SpKial prograM available (cooperative education, adult basic 

edwition, foreign stady, ROTC, etc.) 

65. Library and leaning cHter facilitiM inforiation 

66. Support servicH available (counseling, learning lab, pliceMnt) _ _ _ 

67. Services for handicapped (hcvsing aid building access, 

designated pirUng, interpreters for the deif, tutors, 
leaning aids, counseling, registration assistance, 

trMiportitiM for the Mhility iipaired) _ ^ 

68. Intercollegiate sports offered (by sex and availability of 
Kholvshipt) 

61 Accreditation inforiation 
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U [ ".fy Mt i on Cottossiis Stlitd to Hajor Sourcjs of FinMCia l 4id ud Scholarjhijs 
H.B.I For this SKtioii, you wst chKk MTIOML, HYS, « ABLE 2 tisfs for Mch itw. 

riNMCIM. AID SCHOLARSHIPS (HOT 

ilOVIKmSlI SPOHSOKED) &N'l OR C0LLE6E) 

mm m ABLE NATIOHAL NYS ' A§LE 

70. Halt 

71. Eligibility requiretcnts 

72. Application procedures and deadlina 

73. Size or basil of award 

74. NaM and address of contact person _ 

75. Hov to obtain additional inforiation 

III- SYSTEM DESIGN CONSIDERATIONS 

vu-e w*^'' following elements of system design, please check 

YtS if your system currently has this feature, NO if it does not. 

s!5IE!!Dgisi!F»i!SES S cohhehts 

1. Interactive structured searrh oi occupations by such elenents as 
interests, teiperaients, nu?i«, and skills previously acquired 

2. Ability for users to query the systci as to uby spKific 
occupations did not appear on their lists 

3. Ability to change ansuers in the search and generate lists of 
occupations based on the cka»);ed ansver or answers 

4. DirKt Kcess to inforiation about occupations 

5. Direct access to inforiation about sKoadary areas of 

concentration 

6. Direct access to inforiation about post-sKondary educational 

and training prograis 

7. Interactive structured sNrcb of educational institutions _ 

B. DirKt accMS to inforiition about educational institutions 

Ability to latck institvtiois and educational prograis 

10. Ability to coipare tw or lore educational institutions on the 
Kreen at one tine 
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System Name g 



YES NO COHHEN^ 

U. Liaks to pablifhcd asmsmt instruMnts 

12. Ability to us* occupational characteristics to sort for 

siailar Kcupations 

13. Intnactivt structured search of financial aid inforution 

H. Ability of clients to valk avay froa a session at the coiputer 

with a printed rKord of searches and inforiation rKeived 

13. Ability to create rKords for neu and eierging occupations 
and training progrus 

IV. SYSTEM SUPPORT MATERIALS AND TRAINING 

fo'' each of the following, please check YES if you provide the 
materials or service with your system, NO if you do not. 

!KIiiI«lS Mffi IRA1N1N8 Y|S n|| cDWOTS 

I. Counselor iipleMntation handbooks 

2. Lesson plant or guides for teachers 

3. User handbooks 

4. Training for counselors in systei delivery 




5. Training for labor larket and educational inforaation analysts 

assKiated vith the state or local systea 

6. Training for aanagers or dirKtors of the state or local systens 

V. LENGTH or SESSION 

Approximately how much time does it generally take for 
average high school students to complete a session in which t ey 
use the occupational exploration strategy or strategies and 
generate a list of occupations meaningful to them? 

VI. MACHII4E8 AND MEDIA 

1. List all the time-sharod and microcomputer versions of 
your system currently available and supported by your 
organization. 

i M major uses for any New York State career 

information delivery system will be in classroom settings in 
grades 7 through 9. Describe any delivery media currently 
available and supported by your organization which wouM 
facilitate classroom delivery of occupational exploration and 
occupational information retrieval. 
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in 



System Name ^ 

VII. COMPREHENSIVENESS OF INFORMATION 

°I concerns expressed by respondents of the surveys 

and by guidance personnel interviewed during the site visits was 
the accuracy and comprehensiveness of the information contained 
a computerized system. To help in addressing this issue, please 

Jhe JoU^CJng!*'' ""^ = ' ^ P-^^"*-* 

1. All information which users would obtain if they used the 
Therlpis"*^^ *° information about the occupation Physical 

*° occupation Physical Therapist which 

would keep that occupation on the list of a client using your 
systems occupational structured search. 

3. All information which users would obtain if they used the 
direct access route to information about the post-secondary 
program of study Computer Programming. 

4. All information which users would obtain if they used the 

f *o information about the post-secondary 
institution Stat<» University of New York at Albany. 

VIII. RESEARCH AND EVALUATION STUDIES 

Please identify and describe briefly any studies of your 
system related to the following: 

A. Validity and/or reliability of the instruments used to 
generate clients' occupational lists; 

B. Validation of the occupational data; 

C. Use of the system in Junior high school and/or high 
school classroom or group settings; 

studies to those related to your system as utilized in 
a state-based career information delivery system. 

IX. ADDITIONAL FEATURES — OPTIONAL QUESTION 

If there are any additional or special features of your 
systems which you feel have not been adequately described by 
answering the questions in this request for information, please 
describe them. ' *^ 

X. COSTS 

Assume the following-- 

A. Your system is York State career information delivery 
system. ^ 

B. You are delivering your basic systea. of occupational and 
educational information without any additions of data or files. 

r.J, y**'' delivery, there are 100 MS-DOS (IBM 

compatible) Junior and senior high school sites; in the second 
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System Name jq 

year, lOO similar sites are added? and in the third year, you are 

tt.ii:ii.i:t iLMtii] ^^^^^ 

^' ^ft^ system for each of the 

three years? Provide a work sheet to show the breakdown of costs 
for such Items as initial and subsequent license fees to the 
State; per site license fees; other software fees; training of 
counselors; counselor implementation handbooks @ 1 per site per 

i2**""'fr *° system @ 1 per site per year; user handbooks 

e 500 per site per year; and any other costs needed so that the 
system can be fully implemented, delivered and used. 

2. If the State provided data on entry level wages, 
employment outlook, and licensing requirements for 600 SOC based 
occupations, what would be the total costs for each of the three 
years for your organization to enter and maintain that information 
in your system and deliver it to the sites described above? 
Provide a breakdown of these costs by data entry costs to the 
State; by site use costs and/or other costs as applicable to the 
practices of your organization. 

3. If the State wanted to enter the data in your system, 
would you provide the documentation necessary? What would be the 
cost of this option for each of the three years? Provide a 
breakdown as in 2. 

created an entirely new file which listed 
350 JTPA program providers with information on the area and 
clients served and with links to the relevant occupational 
descriptions in your file. 

4. What would be the total cost to the state ;\->r your 
system to incorporate and deliver this file over the the three 
years described above? Provide a breakdown as requested in 
question 2. 

5. Describe your policies concerning copyrights and use of 
state modification'- to your system. 

"'^^ other systems which will meet the needs of 

New York as described in the directions to this RFI. If you do. 
f fj* describe them briefly and give the cost of utilizing them 
in the three year scenario. 

N.B. : Please be sure that all costs provided are accurate and 
realistic. While this is not a formal bid, it is expected that 
costs over the next three years will be consistent with 
those provided herein. 



Authorized Signature Date' 
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Dear 

The State of New York is conducting a study to evaluate the 
current use of computerized career information delivery systems 
in order to plan for their future use in the state. A letter 
from Assistant Commissioner Kadamus verifying this information is 
included in this mailing. Your system is one of those selected 
for participation in the final stages of the study. 

BACKGROUND AND ACTIVITIES TO DATE 

Several major activities have already taken place as part of 
this study. These include (1) a review of the literature; (2) an 
Information Validation Survey completed by national and State 
experts in the fields of career guidance and occupational 
education; (3) Surveys of User Satisfaction sent to principals, 
counselors, parents, and students at a sample of schools using 
each of the systems; and (4) on-site visits and interviews with 
counselors and administrators. Samples of the survey instruments 
are included in this package for your information. 

OUTLINE OF REQUIREMENTS FOR PARTICIPATION 

Your participation is requested in three major activities, 
which are outlined below. Acceptance of this invitation 
indicates your willingness to participate in all three 
activities. They are: 

1. Written response to a structured request for 
information . 

2. Presentation and demonstration to a System Review Panelc 

3. Review of the system and its materials by the 
Occupational Education Civil Rights Coordinating Unit. 
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1. Written Response To A Structured Request For Informat ion. 
A request for information (Kbl) regarding your~system v/ill be 
sent to you in May with a return three weeks from date of 
mailing. The kinds of information which will be requested 
include such items as implementation and utilization, content, 
availability on a variety of machines, flexibility in various 
settings, costs and fees, etc. 

2. Presentation and Demonstration To A System Review Panel. 
The presentation and demonstration will take place on Tuesday, 
June 24, 1986, from 8:00 am to 5:00 pm at the Ramada Inn on 
Western Avenue in Albany, New York. The System Review Panel will 
consist of approximately fifty people representing the SOICC 
Statutory Committee, the SOICC Advisory Committee, the major 
providers of Occupational Education, the Education Information 
Centers, the Office of Occupational and Continuing Education, 
teachers, counselors, student leadership groups, and others 
deemed appropriate. 

Because of time limitations, the day will be very 
structured. In the morning, you will give a 15 minute speech on 
predetermined topics related to your system. These topics will 
be be sent to you in May along with tlia RFI . The morning's 
presentation will be in a theatre style. 

In the afternoon, you will provide a physical demonstration 
of your system. Members of the System Review Panel will 
circulate among vendors. Each System Review Panel member will be 
required to review every system. Therefore, you may anticipate a 
30 minute demonstration to each of six or seven groups. Each 
CIDS will be provided with a table near a 3 pronged electrical 
outlet. You will be expected to bring your own equipment and a 
power strip if needed. 

The System Review Panel will have rating sheets and will 
rate each system based on the presentation and demonstration. It 
should be stressed that the systems are not in competition with 
each other, but are being measured against a set of criteria. 

3. Review of the System and Its^ Materials by the Occupa tional 
Education CTviTirights CoorHinating UnTTTOECRaiT . The OECRCU 
nas expressed some concern over the compliance of systems with 
civil rights laws and regulations. A separate review of each 
system to ascertain any areas of possible non-compliance will be 
conducted by this Unit's staff. 

You will demonstrate your system to the OECRCU on June 25 in 
their offices. No other vendor will be present at your 
demonstration . 
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To prepare the OECRCU staff, please send one copy 
of the following printed materials: 

User's Manual 

Counselor's Manual 

Brochure describing your system 

Description of any system components for uses related to 
special populations and/or equity issues (optional). 

To: Paul Tonetti, Associate 

Division of Civil Rights and Intercultural Relations 
Room 471, Education Building Annex 
New York State Education Department 
Albany, New York 12234. 

By: May 22, 1986. 

ADDITIONAL INFORMATION 

Rooms have been set aside at the Ramada Inn at the State 
rate of $39 per person per night. This includes a full 
breakfast. The Ramada Inn, should you be arriving by train, bus, 
or plane, has. a courtesy van available until 11 pm. Local 
driving directions are included. 

With the exception of a lunch provided on the 24th, all 
expenses are to paid by the vendors. However, it should be noted 
that there are no exhibition hook-up or demonstration charges for 
the vendors. 

WHAT TO DO NOW 

We realize how tight schedules are at this time of year, but 
New York State really needs this information to move on things in 
the fall. We, therefore, hope that you will find the time to 
cooperate in this project. Please: 

1. Return the enclosed response form no later than May 21, 
1986. 

2. Send the identified materials to the OECRCU immediately. 

3. Make your travel reservations. 

4. Plan to attend the pre-dinner vendor welcome, "A Bit Of A 
Byte", on June 23 from 6:00 to 8:00 pm at the Ramada Inn. 

Joyce Kinnison and I look forward to seeing you. If you 
have any questions, please call me at 212/794-1098. 

Sincerely, 



Deborah Perlmutter Bloch 
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RESPONSE TO CIDS EVALUATION DEMONSTRATION 



I agree to participate in the New York State evaluation of CIDS 
as outlined in the letter of May 13th by Deborah Perlmutter 
Bloch. I understand this includes all of the following: 

1. Written response to a structured request for information. 

2. Presentation and demonstration to a System Review Panel. 

3. Review of the system and its materials by the Occupational 
Education Civil Rights Coordinating Unit (OECRCU). 

I understand that all expenses which will be incurred are the 
responsibility of my organization. 

NAME OF CIDS 



AUTHORIZED SIGNATURE 



DATE 



NAME OF PARTICIPATING REPRESENTATIVE 



ADDRESS OF REPRESENTATIVE 



HOME TELEPHONE NUMBER 

OFFICE TELEPHONE NUMBER 



PLEASE COMPLETE AND RETURN BY MAY 21, 1986 TO: 

Dr. Deborah P. Bloch 
J»ost Office Box 20369 
Cherokee Station Post Office 
New York, NY 10028-9991 



ERIC 



Appendix VI 
SYSTEM REVIEW PANEL: 
INSTRUMENT AND RELATED MATERIALS 



A. Letters of Invitation and Response Forms: 

1. SOICC Members 

2. SOICC Advisory Committee and Other Selected State Experts 
in Guidance and Occupational Education 

3. Selected Major Providers of Occupational Education 

4. Selected Student Leadership Programs 

B. Letter of Instructions to Vendors 

C. System Review Rating Instructions and Forms 
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Deborah F^rlmutter Bloch, Ph.a 



MEMORANDUM 
DATE: May 14, 1986 
TO: SOICC Members 

FROM: Deborah Perlmutter Bloch, Ph. D. 
RE: 



Invitation to participate in the New York State Education 
Department^Evaluation of Career Information Delivery Systems 

As you know, the State of New York is conducting a study~to~~ 
evaluace the current use of computerized career information 
delivery systems (CIDS) in order to plan for their future use in 
e?f ! ^^Zi. cordially invited to participate in the final 

stage of the evaluation of selected CIDS. 

The evaluation will take place from 8:00 am to 5:00 pm on 
Tuesday, June 24, 1986 at the Ramada Inn on Western Avenue in 
^^r^li^^^^ ■^^V^' It^ill consist of two parts. In the morning, 
each CIDS will present orally in response to pre-determined 
questions. After lunch, all the CIDS will demonstrate their 
svstems. The members of the System Review Panel be provided with 
rating sheets and will be expected to rate each system. A 
continental breakfast and buffet lunch will be provided. 

Because of the highly structured nature of the evaluation, 
It IS essential that all participants attend for the entire day. 
\ r^JLll '"S"^ convenient for you, please feel free to designate 
a member of your staff to represent you on the System Review 
fanei. it you choose to send a representative, we will of 
course, be delighted to have you observe any part of the day's 
proceedings. ^ 

.n, ''^P^esentative must be able to register between 8 and 9 

am, begin work by 9 am, stay for a buffet lunch, and complete the 
review of demonstrations scheduled to end at 5 pm. ^"""P^^^^ ^'^^ 

Please complete the attached response form and return it no 
later than May 28, 1986 in the enclosed envelope. "° 

T 1 l^y°^ have any questions, please call me at 212/794-1098. 
I look forward to seeing you on June 24th. 
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RESPONSE TO THE SYSTEM REVIEW PANEL INVITATION 

NAME 



TITLE 



AGENCY 



I' participate as a member of the System Review Panel 

from 8:00 to 5:00 on June 24, 1986? 

Yes No 

2. If you cannot paticipate as a member of the System Review 
Panel, will you be sending a representative of your agency' 
Because of the nature of the competitive rating process, he or 
she must register between 8:00 and 9:00 am and attend from 
9:00 am to 5:00 pm. 

Yes No 



NAME OF REPRESENTATIVE 



TITLE OF REPRESENTATIVE 

WORK ADDRESS OF REPRESENTATIVE 



TELEPONE NUMBER OF REPRESENTATIVE 

PLEASE COMPLETE AND RETURN IN THE ENCLOSED ENVELOPE BY MAY 28, 

^ 7 O w 1 • 

Dr. Deborah P. Bloch 
Post Office Box 20369 
Cherokee Station Post Office 
New York, NY 10028-9991 



ERJC 257 



CIDS Evaluation, Appendix VI, 4 
Deborah Bsrlmutter Bloch, PhD 



MEMORANDUM 
DATE: May 14, 1986 



FROM: 

^?E: 



TO: Members of the SOICC Advisory Committee and Other Selected 
btate Experts in Guidance and Occupational Education 

Deborah Perlmutter Bloch, Ph, D, j&<fi&^ 

Invitation to participate in the New York State Education- 

-f?!!!.!'!"!.!!!!"!!!!" °^ career Information Delivery Systems 

The State of New York is conducting a study to evaluate'the'" 
?C?S?f -"'^ f computerized career information delivery systems 
(CIDS) in order to plan for their future use in the state, A 
letter from Assistant Commissioner Kadamus verifying this 
mfoi-matior. is included in this mailing, 

PaneI^?or^?h^ been recommended as a member of the System Review 
TM. ?^ final stage of the evaluation of selected CIDS. 

This evaluation will take place from 8:00 am to 5:00 pm on 

Ilblny^New^^ork.' ^^""^"^^ °" Western Avenue in 

*.«oh JfnQ^''^^"^"°" "^i^^ consist of twc nrts. In the morning, 
eacn CIDS wili present orally in respor to pre-determined 
questions. After lunch, all the CIDS will demonstrate JSIir 
systems. You will be provided with rating sheets and will be 
expected to rate each system. 

T A continental breakfast and buffet lunch will be provided 
J^^? i ''^^\ ?f '-eimbursed for your travel by rail or* 

R^Jf Tn^^-'\'fK°^ -"^^^ P^"^ ^ room at the 

the 2?rd Ld"9AiS^"V" -^""^ 2^"^' ^i"""^ to $9,50 on 

thP llli 24th, No room reimbursement will be provided for 

^^?oJhi If travel home is not possible on the 24th, please 
telephone me to discuss tnis situation before you return the 
J^mSS^^? V"' y''"'' information, driving directions to the 

?n" ^^""^ enclosed. If you arrivS by train, call the 

Ramada Inn for a courtesy van pick-up until 11:00 pm, 

less tharrfJ^n'!!^ highly structured nature of the evaluation, 
less than a full day's participation on your part will not be 

lllly^^u r^'c^" t° register between 8 aid 9 am, begin 

work by 9 am, stay for a buffet lunch, and complete the review if 
demonstrations scheduled to end at 5 pm. Please note: 
Sly- wequiremeiJ! P'-^vided only to those who fulfill the full 
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We realize this is a busy time for you. Nevertheless, 
because of the importance of this study to New York State, we 
urge you to attend. No expertise in computers is required. You 
have been selected on the basis of your knowledge of occupational 
education or guidance needs. 

Please complete the attached response form and return it no 
later than May 28, 1986 in the enclosed envelope. 

If you have any questions, please call me at 212/794-1098, 
I look forward to seeing you on June 24th. 
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MEMORANDUM 
DATE: May 14, 1986 

TO: Members of the SOICC Advisory Committee and Other Selected 
State Experts in Guidance and Occupational Education 

FROM: 
RE: 



Deborah Perlmutter Bloch, Ph. O.,^^^ 

Invitation to participate in the New York State Education 
.-fff!!!!!"! Evaluation of Career Information Delivery Systems 



The State of New York is conducting a study to evaluate the 
?r^nc\ ."^^ ^ computerized career information delivery systems 
(CIDS) in order to plan for their future use in the State. A 
letter from Assistant Commissioner Kadamus verifying this 
information is included in this mailing. 

Por,.!^?" been recommended as a member of the System Review 

Panel for the final stage of the evaluation of selected CIDS. 
This evaluation will take place from 8:00 am to 5:00 pm on 
Tuesday, June 24, 1986 at the Ramada Inn on Western Avenue in 
Albany, New York. 

P^rh J?nc:^''f}y^*'^°" ""l^^ consist of two parts. In the morning. 
on^^^^^ will present Of ally in response to pre-determined 
questions. After lunch, all the CtDS will demonstrate their 

ovoooJ^; Provided with rating sheets and will be 

expected to rate each system. 

T A continental breakfast and buffet lunch will be provided. 
r^r^'fti ""H^ reimbursed for your travel by rail or 

car (at $.23 per mile plus tolls). Travel directions to the 

?nrfnr }^ ^"^^^ train, call the Ramada 

Inn tor a courtesy van pick-up. 

locc ?J""^Vf/il^ structured nature of the evaluation, 

i^e^^?^" "^^^i^ participation on your part will not be 

^ t u a ■""^'^ ^^^^ ^° register between 8 and 9 am, begin 

work by 9 am, stay for a buffet: lunch, and complete the review^Jf 
demonstrations scheduled to end at 5 pm. Please note- 
day's"Jequiremeits.^' provided only to those who fulfill the full 

We realize this is a busy time for you. Nevertheless, 
because of the importance of this study to New York State we 
urge you to attend. No expertise in computers is required. You 

e^duca^^^n" or'|S^d1n?2 ^^^d^!^'^ °' °' occupational 

Please complete the attached response form and return it no 
later than May 28, 1986 in the enclosed envelope. If you haie any 
questions, please call me at 212/794-1098. T look f6rSa?§ to^ 
seeing you on June 24th. 
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MEMORANDUM 
DATE: May 14, 1986 

TO: Members of the SOICC Advisory Committee and Other Selected 
State Experts in Guidance and Occupational Education 

FROM: Deborah Per' utter Bloch, Ph. D. Sff^ 

^^^^^l^'^^'l 5° participate in the New York State Educata.un 
Department ^valuation of Career Information Delivery Syst 



ion 

ems 



The State of New York is conducting a study to evaluate'the 
?r?Slf computerized career information deUve^y systems 

(CIDS) in order to plan for their future use in the StatI A 
letter from Assistant Commissioner Kadamus verifying this' 
information is included in this mailing. ^ 

PanPl^?or^fho ^een recommended as a member of the System Review 

?hri e^°lu^M o^"^ 1 '^T °f evaluation of selected CIDS. 
Ihis evaluation will take place from 8:00 am to 5-00 om on 

each CIDS^wn Wr°" consist of two parts. I„ the morning, 

A continental breakfast and buffet lunch will be provided. 

l^ss ?hpr!%°f/!;^ ^^^^^y structured nature of the evaluation, 
less than a full day's participation on your part will not be 
possible You must be able to register between 8 and 9 am begin 
work by 9 am, stay for a buffet iSnch, and complete the ?eview^of 
demonstrations scheduled to end at 5 pm. '"P^^'^^ cne review of 

We realize this is a b- y time for you. Nevertheless 
because of the importance o. this study to New York Statlwe 
urge you to attend. No expertise in computers is reqS?Jld You 

I looJ'fo^?:;a!;d^\%^L^e?ng"JiroA f^^^lC.^} 212/794-1098. 
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RESPONSE TO SYSTEM REVIEW PANEL INVITATION 



NAME 

SCHOOL OR AGENCY 
ADDRESS 



TELEPHONE NUMBER 

I will 

I will not 



participate in the evaluation of career inf oraation delivery 
systems on June 24, 1986. ^ 

I understand that because of the nature of the comDetitive raring 
process, I must register between 8:00 and 5:00 am and attend from 
y.UU an to 5:00 pm. 



Signature" 



RETURN IN THE ENCLOSED ENVELOPE BY MAY 28 
1986 TO I ' 

Dr. Deborah P. Bloch 
Post Office Box 20369 
Cherokee Station Pose Office 
New York, NY 10028-9991 
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Deborah ftrlmutter Bloch, Ph.D. 



MEMORANDUM 
DATE: May 13, 1986 

TO: Selected Major Providers of Occupational Education 
FROM: Deborah Perlmutter Bloch, Ph. 

RE: Invitation to Participate in the New York State Education 
Departinenr_Review_of_Career Information Delivery Systeig 

The State of New York is conducting a study'to'evaluate'the 
f computerized career information deUvl^y systems 
(CiDb) in order to plan for their future use in the State A 
letter from Assistant Commissioner Kad.mus vei^fying thJs' 
information is included in this mailing. "y^"8 ^^^^ 

Your agency has been selected as one of the maior DroviHpr^ 
?L°rT'i-"^^ education to participate in the fJni? stige ol 
i^r.^^'o selected CIDS, Please select a counselor 

servfo^; ^^^^i^^^ P™el services to 

The evaluation will take place from 8:00 am to 5-00 nm on 
I^bany^New^Yo k' ?n''^ Ra.ada Inn on Western A^eSu^in" 

each CIDS win n;pJi/ \?°"''''^ °^ P"'^^' I" the morning. 

Questions Ai^Sr i ^^''^H^ '""ponse to pre-determined ^ 
svltems Thp ^plhi""^^ ll^ l^^ ^^^^ ^^^^ demonstrate their 

with ritini ^hpp?^ ^^^^^^ ^ill be provided 

with rating sheets and will be expected to rate each system. 

Tn breakfast and buffet lunch will be provided 

arla^^ZS ^^r 1^ ^l^^^^^' 
LTf^o%^rc?u%%:^f :in^^picJ5ur ""^^ ?a^^^°h^^%mada 

less ?hpn"f%°f/^^ ^^^^^y structured nature of the evaluation, 
less than a full aay's participation on the part of your 

r%'g'[s"lf bet^e'en's'and^'' acceptable. He o? she .u^t'Je able to 
but f pt 1 nn^£ ^^8in work by 9 am, stay for a 

to end it 5 om^ pfp:?i''%'^" °^ demonstrations scheduled 

to th"^sf whV?ulf^}rt\%"?i?i dljt"r%^^:r?e^rii! 

^tpff"^ IT^^lt ^"^y time of year for you and your 

New"ork Itate ifn' ^""^ importance If th?s IZly to 

New YorK fttate, we urge your participation. Exoertise in 

Sua%"wit\"?L'''^yH'''^'- '^r'^ ^^^-^ - rep?e'se^^?a% ve 
tamiliar with the guidance needs of the students you serve. 

later'thl^Mav'"28t£^n^^^"^"?"^ response form and return It no 
t7,lZl^ ?y ^" the enclosed envelope. If you have snv 

questions, please call me at 212/794-1098. ^ 
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Deborah Berlmutter Blpch, Ph.D 



FROM 
RE: 



MEMORANDUM 
DATE: May 13, 1986 

TO: Selected Major Provide-^ of Occupational Education 
Deborah Perlmutter Bloch, Ph, D, fQf^ 

Invitation to Participate in the New York State Education 
Department Review of Career Information Delivery Systems 

The State of New York is conducting a study to evaluate the 
^"^VVl^ °^ computerised career information delivery systems 
(CIDS) in order to plan ior their future use in the State. A 
letter from AssistatJt C -omissioner Kadamus verifying this 
information is included in this mailing. 

Your agency has b^en selected as one of the major providers 
of occupational education to participate in the final stage of 
the evaluation of selected CIDS, Please select a counselor, 
director of guidance, or director of pupil personnel services to 
serve on the System Review Pane], 

The evaluation will take pXace t:c^ 8:00 am to 5:00 pm on 
Tuesday, June 24, 1986 at the Ramada I'm on Western Avenue in 

^Vr^c.^"^ ^.V^' "^^^^ consist of two parts. In the morning, 

e&ch CIDS will present orally in response to pre-determined 
questions. After lunch, all the CIDS will demonstrate their 
systems. The members of the System Review Panel will be provided 
with rating sheets and will be expected to rate each system. 

A continental breakfast and buffet lunch will be provided. 
In addition, you will be reimburses for your travel by rail or 
car (at the rate of $.23 per mile plus tolls), for a room at the 
Ramada Inn in Albany on June 23rd, and for dinners up to $9.50 on 
the 2Jrd and 24th. No room reinbursement will be provided for 
the 24th. If travel home is not possible on the 24th, please 
telephone me to discuss this situation before you return the 
response form. For your information, driving directions to the 
Ri--iida Inn have been enclosed. If you arrive by train, call the 
Ramada Inn for a courtesy van pick-up until 11:00 pm. 

Because of the highly structured nature of the evaluation, 
less r?an a full day's participation on the part of your 
representative will not be possible. He or she must be able to 
register between 8 and 9 am, begin work by 9 am, stay for a 
buffet lunch, and complete the review of demonstrations scheduled 
to end at 5 pm. Please note: reimbursement will be provided only 
to those who fulfill the full day's requirement. 
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<=^.ff ^ ? ''"^y ^^""^ °^ y^^'' fo'' and your 

vA.v ^^^^"^^ °f the importance of this study to 

New York State, we urge your participation. Expertise in 
computers is not required. Please select a -epresentative 
tamiliar with the guidance needs of the students you serve. 

response form and return it no 
later than May 28th in the enclosed envelope. If you have any 
questions, please call me at 212/794-1098. 
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RESPONSE TO SYSTEM REVIEW PANEL INVITATION 



SCHOOL OR AGENCY 
ADDRESS 



PRINCIPAL/DIRECTOR 



We will 



We will not 



participate in the evaluation of career information delivery 
systems on June 24, 1986, 

The individual named below will represent the school or agency, 
I understand that because of the nature of the competitive rating 
process, he or she must register between 8:00 and 9:00 am and 
attend from 9:00 am and 5:00 pm. 



NAME OF REPRESENTATIVE 



WORK ADDRESS OF REPRESENTATIVE 



TELEPHONE NUMBER OF REPRESENTATIVE 



Authorized Signature' 



PLEASE^COMPLETE AND RETURN IN THE ENCLOSED ENVELOPE BY MAY 28, 

Dr, Deborah P, Bloch 
Post Office Box 20369 
Cherokee Station Post Office 
New York, NY 10028-9991 
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Deborah Fferlmutter Bloch, Ph.D 



MEMORANDUM 
DATE: May 14, 1986 

TO: Selected Student Leadership Programs 
FROM: Deborah Perlmutter Bloch, Ph. D. f^if^ 

RE: Invitation to participate in the Nev; York State Education 

Department Evaluation of Career Information Delivery Systems 

The State of New York is conducting a study to evaluate the 
current use of computerized career information delivery systems 
(CIDS) in order to plan for their future use in the State. A 
letter from Assistant Commissioner Kadamus verifying this 
information is included in this mailing. 

Your occupational education student leadership program has 
been selected to participate in the final stage of the evaluation 
of selected CIDS. Please select a student leader and faculty 
member from your program to serve on the System Review Panel'. 
(The faculty member with both accompany the student and be a 
member of the panel.) 

This evaluation will take place from 8:00 am to 5:00 pm on 
Tuesday, June 24, 1986 at the Ramada Inn on Western Avenue iu 
Albany, New York. It wilT consist of two parts. In the morning, 
each CIDS will present orally in response to pre-determined 
questions. After lunch, all the CIDS will demonstrate their 
systems. Members of the System Review Panel will be provided 
with rating sheets and will be expected to rate each system. 

A continental breakfast and buffet lunch will be provided. 

Because of the highly structured nature of the evaluation, 
itss than a full day's participation on the part of your 
lepresentatives will not be possible. They must be able to 
register between 8 and 9 am, begin work by 9 am, stay for a 
buffet lunch, and complete the review of demonstrations scheduled 
to end at 5 pm. 

We realize this is a busy time of year in the schools. 
Nevertheless, because of the importance of this study to New York 
State, we urge your participation. Expertise in computers is not 
required. 

Please complete the attached response form and return it no 
later than May 28, 1986 in the enclosed envelope. If you have 
any questions, please call me aL 212/794-1098. 
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RESPONSE TO SYSTEM REVIEW PANEL INVITATION 

SCHOOL OR AGENCY^ 

ADDRESS_ 

DIRECTOR/PRINCIPAL_ 

We will 



We will not 



participate in the evaluation of career information delivery 
systems on June 24, 1986. 



* * * 

If participating: 

The student leader and faculty member named below will represent 
the program. I understand that because of the nature of the 
competitive rating process, they must register between 8:00 and 
9:00 am and attend from 9:00 am and 5:00 pm. 

NAME OF STUDENT LEADER 



NAME OF FACULTY MEMBER 



WORK ADDRESS OF FACULTY MEMBER 



TELEPHONE NUMBER OF FACULTY MEMBER 



* * * 



AUTHORIZED SIGNATURE 



PLEASE COMPLETE AND RETURN IN THE ENCLOSED ENVELOPE BY MAY 28 
1986 TO: ' 

Dr. Deborah P. Bloch 
Post Office Box 20369 
Cherokee Station Post Office 
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Deborah ReHmuttJerBloch,Ph.D 444^»i»<iSum*NewYbA.Nrn(m (jiDm-togt 
DATE: May 19, 1986 

TO: Selected Career Information Delivery Systems 

FROM: Deborah Perlmutter Bloch 

RE: Oral Presentation on June 24, 1986 



will c^.rJ^l J?rning of June 24, 1986, your system's repre, entative 
will give a fifteen minute oral presentation to the System Review 
Panel for the New York State evaluation of computerized careir 
information systems. ^ career 

Please direct that presentation to the following four topics: 

1. An overview of your system 

2. The philosophy underlying your system 



3 



The use of your system in classroom or group settings at the 
junior or senior higti school level 

4. The major contribution your system makes to an individual's 
career planning. 

Just as a reminder, the other requirements for your 
participation in this study are: 

-Demonstrations of your system to the System Review Panel i the 
f ^'^^^24th; (Be sure to bring user handbooks and 
materials for e:j^lanation by the panel.) 

'S?ah?r?^"Sr °5.^°".^ ^y^*'^'" ^° Occupational Education Civil 
Rights Coordinating Unit on the day previously specified; 

-Completion of the enclosed Request for Information. 

If you have any questions on the presentation, 

(TlT)^llT^'^Vo ^^^""t for information, please call me at 
Kill) 794-x092 or Joyce Kinnison at (919) 469-0081. 
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SYSTEM REVIEW PANEL RATING INSTRUMENT 
CAREER INFORMATION DELIVERY SYSTEMS PRESENTATION AND DEMONSTRATION 

JUNE 24, 1986 

NAME 



TITLE 



SCHOOL, AGENCY OR PROGRAM 



The presentation and demonstration of computer-based career 
information delivery systems to the Systo-, Review Panel is one of 
the culminating activities in the study of these systems being 
conducted by New York State. The overall purpose of the study is 
to identify the system or systems best suited to meet the various 
needs of individuals and programs. These include: 

1. Implementation of the three occupational education 
curricula "Home and Career Skills," "Introduction to 
Technology," and "Introduction to Occupations;" 

2. Implementation of guidance plans for junior and senior 
high school students; 

3. Career and educational planning by individuals in such 
groups as junior and senior high school students, out-of- 
school youth, community and technical college students, 
handicapped workers and other adults. 

In the course of this study, a great deal of information has 
been gathered on the components of the selected systems, the 
comprehensiveness and accuracy of the data within the systems, 
the reliability and validity of the instruments used for 
exploration within the systems, and the costs of the systems. 
What cannot be gathered through surveys and requests for 
information is how it feels to use a system, how appealing or 
interesting it is to a student or client, or what the holistic 
experience or gestalt of using the system is. That is what you 
are being asked to evaluate. 



You may feel that the items on which you will b;i asked to 
rate the systems are subjective. They would be if only one 
person rated the systems. However, each of you is an expert in 
a particular area of occupational education or counseling. 
Therefore, the sum total of your expert ratings becomes an 
objective evaluation of the usefulness and appropriateness of the 
systems. 

When all the information gathered during this study is 
analyzed, your ratings will be combined with the results of the 
vendors responses to request for information, the Surveys of 
User Satisfaction, and the site visits to formulate the 
final conclusions and recommendations. 
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4, In making your ratings, you must circle one number for each 
item, xf you circle more than one number or do not respond to an 
Item, your entire rating form will be invalid. Feel free to 
comment at length on specific features or qualities of a system 
in the blank spaces provided. 

5, When you are rating a system, judge it in terms of the needs 
and characteristics of the population your school, agency or 
program serves, 

6, Remember that the systems are not being judged against each 
other, but against a set of standards, 

7, After you have finished rating a system, move to the next 
sequentially numbered table, 

8, When you have finished rating all six systems, turn in your 
entire rating booklet to Deborah Bloch or Joyce Kinnison at the 
door. 

Thank you for your assistance xn this process. 
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AGENDA 



3;:' - 


9:00 


an 


-^^^e-*- ^ »^ -Lun ana conLinenLai Dreakrast 


^ . 00 - 


9 • 30 


A m 
a Hi 


welcome ana orientation 

Questions and answers on the rating process 


9:30 - 


10:30 


am 


First three oral presentations by vendors 


10:30 - 


10:45 


am 


Brief break 


lU : - 


11 :45 


am 


Final three oral presentations by vendors 


12:00 - 


12:45 


pm 


Buffet lunch in the Loft Room 


12:45 - 


5:00 


pm 


Vendor demonstrations and system review panel 
ratings in the Guilder Room 



DIRECTIONS 

1. In the morning, each vendor will make a 15 minute 
presentation on the four following topics. 

A. An overview of the system; 

B. The philosophy underlying the system; 

C. The use of the system in classroom or group settings at the 
junior or senior high school level; 

D. The major contribution the system makes to an individual's 
career planning. 

Space has been provid'.<: the rating book for you to take notes, 
on each presentation. T!. ^ notes are to aid you in making your 
ratings. They will n ' considered as part of your rating of 
the system. ^ 

lu ^".f J^s_,3fternoon, all the vendors will have tables at which 
they will demonstrate their systems and user materials. Please 
begin viewing the demonstrations at 12:45 pm. Start at the table 
number on the lower right-hand corner of your booklet. Spend 
approximately 30 minutes at each vendor's table. 

3. After you finish at each demonstration, complete the rating 
for that syscem based on your experience in the demonstration and 
the oral presentation of the morning. For each system, there are 
15 items to be rated on a scale of 1 to 5, by circling the number. 

5=Excellent 

4=Good 

3=Fair 

2=Poor 

l=Atrocious 

Do not hesitate to use the extremes of the scale. If a feature 
is not available at all within a system, circle the 0. 
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NQIii QN IHE QRAL i^RESENTATigN 

FOR 

1. Overvie^w of System Components 



2. Underlying Philosophy 
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Oral Presentation Notes, p. 2 

3. Use of the System in T - assroom or Group Settings 



4. Major Contribution of the System to Individuals' Career 
Planning 
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RATINGS 



FOR 



1. Ease of reading 

2. Clarity of screen display 

3. Appeal of the systei ~ the extent to which it encourages 
client or student use 

4. Relevance of the user handbook to the student's or cl:(<it*s 
point of viett 

5. Appropriateness of the structured search of occupations 

6. Ability of the user to discontinue the search be/ore completing 
it and Mve dirKtly to occupational inforiation 

7. Level of detail of Kcupational inforution 

9. Appropriateness and currency of teriinology in occupational 
inforiation 

9. Clearly defined links between occupational and educational 
inforiation 

10. Appropriateness of the structured search of post-SKondary 
educational institutions 

It. Level of detail of ct)ll«ge and university inforiation 

12. Ability of users to enter the systei at their points of 
readiness or interest rather than at syttei-deteriined entry 
points 

13. Ability to use the systei in group or dassrooi settings 

14. Overall ease of use 

15. Overall rating 

Cornmentg 



S 
S 



s 
s 

s 

3 



s 

5 

3 
5 
S 



3 
3 

3 
3 



3 
3 

3 
3 
3 
3 



Poor *- »'2?isSS jiot 
Available 



2 
2 

2 
2 



2 
2 

2 
2 
2 
2 



Please use the remainder and reverse side of this page to make 
any comments you wish about the system, its contents, its 
usefulness, its strengths, or its weaknesses. 
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Appendix VII 
PROCEDURE AND INSTRUMENTS FOR EVALUATING CIDS 

Procedure for evaluating CIDS 

Rating Guide for Information Components, System Design 
Considerations, and Support Materials and Training 
Rating Guide for Occupational and College Information 
Printouts 

Rating Guide for System Review Panel Ratings 
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A. PROCEDURE FOR EVALUATING CIDS 
The procedure for rating CIDS which was developed for use in 
this study can be used by any person or group wishing to evaluate 
a system against established standards. The standards used in 
this study were validated through an extensive survey process 
described in the report. 

The evaluation consists of three components which are 
handled in a seven-step process. The findings of this study have 
shown that this process can serve as a stro- ». predictor of user 
satisfaction with a CIDS. 

The seven steps are outlined below and associated rating 
instruments follow in this Appendix. 

Step 1; Collect information from vendors of CIDS concerning 
their system's information components, system design features, 
a.id support materials and training. These items are contained in 
the first four sections of the Request for Information (RFI) 
which appears in Appendix V. 

Stji2 2: Collect printouts from the vendors of CIDS which 
contain ipfcrmfitl on about one occupation and one college 
contained in their systems. Specify an occupation and a college 
the CIDS sho'.'ld contain and about which you can get reliable 
information. 3(,Cwion VII of the RFI is designed to collect this. 

Step 3: Plan and conduct the vendor demonstration to a 
System Review Panel which represents groups which use CIDS and 
otter agency and school personnel. This step is discussed in 
detail in the "Methodology" chapter in this study, and the System 
Review Panel rating book can be found in Appendix VI. 
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Step 4: Rate the information which the vendors provided 
in response to the RFI according to the Rating Guide for 
Information Components, System Design Considerations, and Support 
Materials and Training, which follows in this Appendix. Each 
item is worth 1, 2, or 3 points as indicated. (Some of the items 
or. the RFI have not been assigned points. Therefore, the rating 
guide does not Include all items on the RFI.) 

The guide contains a possible 300 points. In order to give 
equal weight to *11 three of the standards for evaluating CIDS in 
this process, the total points received by CIDS as the result of 
this first rating are divided by 300 to obtain a percentage score. 

Ste£ 5; R«te the occupational and college information on 
the printouts provided by the vendors using the Rating Guide of 
Occupation^il and College Information Printouts in this Appendix. 
There is a total of 100 possible points available from this 
^-rtli^.c;. The points received by a CIDS provides a percentage 
score for the comprehensiveness and accuracy of information 
con::ained in the d«ita base . 

If & CIDS printout contains no information on an item, score 
it 0. Hovever, if you are not able to score all items because you 
do not have the information available to judge the CIDS 
printout, subtract the worth of that Item from 100 t" compute the 
highest possible total score. Then divide the score earned by 
the CIDS by that highest possible score. This converts the 
earned score to a perceuJage. 

S-^. i- Rate the results of the System Review Panel. First 
coapute the mean of each item by adding scores of all raters on 
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that item and dividing by the number of rater.5. Then double the 
nean scores on the items to be weighted double according to the 
Rating Guide for S^'atem Re/iew Panels. Finally add all the meam 
for each CIDS for the score of the CIE-S on this portion. 
Ste£ 7: Compute the final score for each CIDS. Add the scores 
obtained as the result of steps 4, 5, and 6. Divide by 3. That 
the final score. Systems scoring less than 60 should not be 
rt:.^f: oved. A final score of 60-697. is passing; 70-797. is 
satisfactory; 80-897. is good; 90-1007. is xcellent. 
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B. RATING GUIDE FOR INFORMATION COMPONENTS, SYSTEM DESIGN 
CONSIDERATIONS, AND SUPPORT MATERIALS AND TRAINING 

Directions for Rating: Assign the points indicated for each item the 
vendors check as part of their system in response to the Request for 
Information. If the vendor checks NYS, give points for both NYS and 
ABLE since if they have information, they are able to have it. 



I™§ . WORTH OF ITEMS 

NATIONAL NYS ABLE 

A^ Information Components Related to Specific Occupations 

1. Occupational description 2 

2* Occupational group or cluster 2 

3* Job duties 2 2 2 

4. Interests (of workers as 

validated in various studies) 2 

5* Aptitudes 2 

6. Temperaments 2 

7. Entry level skills required for job 2 2 2 

8. i^hysical demands 

(amount of weight the worker 

must lift or carry) 2 

9. Pi'ysical act iv ..fries 2 

10. Tools, equipment, etc. 

used on the job 2 

11. EnvironmenLal/work conditions 2 
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IIB!^ WORTH OF ITEMC 



2 

1 1 



NATIONAL m'S ABLE 

12. Hours of work and travel 

required on the job 2 2 2 

13. Industries in which occupations 

occur (places of employment) 2 2 

14. Hiring channels 1 

15. Career ladders and 
advancement opport;,? : ities 1 

16. Occupational outlook 2 2 2 

17. Current and future occupatiotial 
supply and demand 111 

18. Current employment (numbers, by 
occupation by industry) 111 

19. Earnings and benefits 
(beginning, average, range, 

fringe benefits) 2 2 

20. Education/training required 2 

21. Licensing, certification, 
tests, etc. required 2 2 £ 

22. Helpful secondary school subjects 2 

23. Related occupations 2 

24. Related military occupations 1 

25. Sources of additional information 111 

26. Bias-free data base — occ's represent 
entire range of the labor market 2 2 2 
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ITEMS 



— . , J:-?JIlL2L_tX^£ 

27. Bias-free occupat'I titles and contents 2 

28. Occupat-ional classification and 
coding system based on the DOT or 

or SOC 2 
Information Components Related to Labor M«^rket Entry 

29. Job seeking skills 2 

30. Interview techniques 

31. Employment services 

32. Self-employment and 
entreprenet rial information 
Information Components Related to Sf condary 

33. Area of concentration description 



ABLE 



34. Area of concentration group or cluster 

35. Typical course work 

36. Types of learning activities 

37. Skills and knowledge acquired 

38. Related entry level occupations 

39. Related occupations requiring 
further study 

AO. Related post-secondary programs 

Information Components Related to Post-Secondary 

Educati onal and Tra ining Programs 
Al. Program description 2 
A2. Program group or cluster 2 



Areas of Concentration 
1 1 



1 
1 
1 



1 
1 
1 
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ITEMS 



WORTH OF ITEMS 



43. Typical course work 

44. Types of learning activities 

45. Skills and knowledge acquired 

46. Certificates or degrees earned 

47. Normal length of time to 
cr.plete certificate or degree 

48. Related occupations 

49. Related pro,;.'ams of study 

50. Type(s) of institutions 
offering program 



NATIONAL 
2 
2 
2 
2 



2 
2 
2 



NYS 



ABLE 



TWO- AND FOUR- YEAR 
COLLEGES (POST-SEC.) 
NATIONAL NYS ABLE 



VOCATIONAL TRAINING 



NATIONAL NYS ABLE 



Information Components Related to Educational and Trainin,^ 
Institutions 

51. Geographic location 2 1 

52. Type of institution 2 

53. Admission Requirements 
(test scores, application 
procedures, application 
deadline, fees, 

physical eicam, etc.) 2 1 
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ITEMS 



WORTH OF ITEMS 



TWO- AND FOUR- YEAR 
COLLEGES (POST-SEC.) 
NATIONAL NYS ABLE 



54. Name and address of 
admissions contact 
person 1 

55. Cost (in-state and 
out-of state tuition, 
room, board, fees) 2 

56. Housing information 2 

57. Financial aid informatics 
(sources, application 
procedures, application 
deadline, average 

award, etc.) 2 

58. Name and address 
of financial aid 
contact person 1 

59. Education and training 2 
programs offered 

60. Types of certificates 

or degrees awarded 2 

61. Normal length of time 

to complete certificates/ 
degrees offered 2 



VOCATIONAL TRAINING 



NATIONAL yYS ABLE 
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ITEMS 



WORTH OF ITEMS 



TWO- AND FOUR -YEAR 
COLLEGES (POST-SEC.) 
NATIONAL NYS ABLE 



VOCATIOfi.Vi:, 



62. Special programs 
available (cooperative 
education, adult basic 
education, foreign 
study, ROTC, etc.) 2 

63. Library and learning 
center facilit?es 
information 1 

64. Support services 
available (counseling, 
learning lab, 

placement) 2 

65. Services for handicapped 2 

66. Intercollegiate sports 

':fered (by sex and 
. /- liability of 
scholarships) 2 

67. Accreditation 1 



NATIONAL NYS ABLE 



2 
2 



2 
2 



1 
1 



1 
1 
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F\ Information Components Related to Major Source s of 
Financial Aid and Scholarships 



ITEMS 



WORTH OF ITEMS 



FINANCIAL AID 
(GOVERNMENT SPONSORED) 
NATIONAL NYS ABLE 



1 
1 



68. Eligibility 
requirements 1 

69. Application procedures 
and deadline 1 

70. Size or basis 
of award 1 

71. Name and address 
of contact person i 

72. How tc obtain 
addi information 1 

£i Design Considerations and features 

73. Inter;?.-; l .^ ??tr-K'c'ired search of occupations by 
such feies'^^-ts as Interests, temperaments, values, 
and skills previously acquired 

74. Ability for users to query the system as to why 
specific occupations did not appear on their lists 

75. Ability to che^ge answers ir ^ he search and 
generate lists of occupations based on the 
changed answer or answers 



SCHOLARSHIPS (NOT 
GOV'T OR COLLEGE) 
NATIONAL NYS ABLE 



YES 
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WORTH OF ITEMS 

YES 

76. Direct access to information about occupations 3 

77. Direct access to information about secondary 

areas of concentration 1 

78. Direct access to information about post-secondary 
educational and training programs 1 

79. Interactive structured search of iducatiDnal 
institutions 3 

80. Direct access to information about educational 
institutions 3 

81. Ability to match institutions and educational 
programs ^ 

82. Ability to compare two or more educational 
institutions on the screen at one time 1 

83. Links to published assessment instruments 1 
Ability to use occupational characteristics to sort 

for sirailar occupations 1 

85. Interactive structured search of financial aid 
information ^ 

86. Ability of clients to walk away from a session at the 
computer with a pr:lnted record of searches and 
information received 3 

87. Ability co create records for new and emerging 
occupations and training programs 
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■ ^ WORTH OF ITEMS 

H. Support Materials and Training 

YES 

88. Counselor implementation handbooks 3 

89. Lesson plans or guides for teachers 3 

90. User handbooks 3 

91. Training for counselors in system delivery 3 

92. Training for labor market and educational information 
analysts as" related with the state or local system 1 

93. Training for managers or directors of the state 

or local systems 1 
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C. RATING GUIDE FOR OCCUPATIONAL AND COLLEGE INFORMATION PRINTOUTS 

Directions for Rating: Assess the accuracy and comprehensiveness of each 
item on the printouts provided by the vendors and assign the appropriate 
number of pointzs. 



COMPONENT 



EXCELLENT GOOD FAIR POOR ATROCIOUS NONE 



Occupational Information 

I. Bias-free titles and 
descriptions 

Occupational description 
Job duties 

Interests and temperaments 
Aptitudes 

Skills, education, training 
req'd for entry level 
position 

Physical demands/activities 
Working conditions 
Future outlook 
10. Earnings 

II. Related occupations 
12. Places of employment 
College Information 

1. Admission Requirements 

2. Cost (for year specified) 

3. Type of institution 



2. 
3. 
4. 
5. 
6. 



7. 
8. 
9. 



5 
5 
5 
5 
5 



5 
5 
5 
5 
5 
5 
5 

5 
5 
5 



4 
4 
4 
4 
4 



4 
4 
4 
4 
4 
4 
4 

4 
4 
4 



3 
3 
3 
3 
3 



3 
3 
3 
3 
3 
3 
3 

3 
3 
3 



2 
2 
2 
2 
2 



2 
2 
2 
2 
2 
2 
2 

2 
2 
2 



0 
0 
0 
0 
0 



0 
0 
0 
0 
0 
0 
0 

0 
0 
0 
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COMPONENT 


EXCELLENT 


GOOD 


FAIR 


POOR 


ATROCIOUS 


NONE 


4. Housing availability 


5 


4 


3 


2 


1 


0 


5. Information about the 














graduate programs in 














education 


5 


4 


3 


2 


1 


0 


6. Services for handicapped 


5 


4 


3 


2 


1 


0 


7. Intercollegiate sports 


5 


4 


3 


2 


1 


0 


8. Enrollment: # of students 


5 


4 


3 


2 


1 


0 



On the guide for rating occupational and college information 
printouts, rating points are assigned to measure 
comprehensiveness and accuracy. Some sources which can be used to 
determine the accuracy of the occupational information are: SOC 
Career Profiles (National Crosswalk Center, 1985;) Occupational 
Employment and Training Data (U.S. Department of Labor, 1986;) 
Occupational Outlook Handbook , 1986-87 (U.S. Department of Labor, 
1986;) and the Guide for Occupational Exploration (U.S. 
Department of Labor, 1984.) Sources for the college information 
include Che college catalogue and/or information supplied by the 
New York State Office of Higher and Professional Education. 

Some additional guidelines for the rating of occupational 
information follow. 

1. Bias free job titles and descriptions 

5»all information completely bias free 
4»1 statement or title showing bias 
3-2 statements or titles showing bias 
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2«3 statements or titles showing bias 

1- 4 statements or titles showing bias 

0- 5 or more statements or titles showing bias 
Occupational description 

5«description totally accurate and complete 

4- 1 inaccurate statement and/or critical piece of 

information missing 
3-2 inaccurate statements and/or critical pieces of 
information missing 

2- 3 inaccurate statements and/or critical pieces of 

information missing 

1- 4 inaccurate statements and/or critical pieces of 

information missing 
0-5 or more inaccurate statements and/or critical pieces 

of information missing, or description not present 
Job duties, including tools and equipment used, if applicable 
(can be part of the description or separate) 

5- adequate, correct information to understand duties of the 

job 

3- partial information present and correct 
O=information not correct, or job duties not present 

Interests and temperaments 

5»both interests and temperaments present ^nd accurate 
4»both interests and temperaments present, with 1 or 2 

missing or inaccurate 
3-both interests and temperaments present, with 3 or 4 
missing or inaccurate 
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2«either interests or teaiperaments (but not both) pr<=^sent 
and accurate 

l«either interests or temperaments (but not both) present, 

with 1 or 2 missing or inaccurate 
0»either interests or temperaments (but not both) present, 

with two or more inaccurate, or neither interests nor 

temperaments present 
Aptitudes 

5»9 aptitudes listed and coded correctly 

4»7-8 aptitudes listed and coded correctly 

3«5-6 aptitudes listed and coded correctly 

2=3-4 aptitudes listed and coded correctly 

l»l-2 aptitudes listed and coded correctly 

0»all aptitudes incorrect, or aptitudes not present 
Skills, education, and training required for entry level 
positions 

5«all items included and accurate 

4«1 inaccurate or missing item 

3=2 inaccurate or missing items 

2=3 inaccurate or missing items 

1=4 inaccurate or missing items 

0=5 inaccurate or missing items, or category not present 
Physical demands and activities 

5=all applicable physical demands and activities present and 
accurate 

4=1 physical demand or activity not present or inaccurate 
3=2 physical demands or activities not present or inaccurate 
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2- 3 physical demands or activities not present or inaccurate 

1- 4 physical demands or activities not present or inaccurate 

0- 5 or more physical demands or activities not present or 

inaccurate, or physical demands and activities not present 
Working conditions (includes environmental conditions 
and hours of work and travel) 

5-all information present and correct 

4- 1 information item missing or incorrect 

3- 2 information items missing or incorrect 

2- 3 information items missing or incorrect 

1- 4 information items missing or incorrect 

0-5 or more information items missing or incorrect, or 
working conditions not present 
Future outlook. 

5- future outlook statement present and accurate to specified 

time in the future (eg. 10 years, to 1995) 
4»future outlook statement present and accurate without 

specified time in the future 
0-future outlook statement incorrect, or future outlook 

statement not present 
Earnings (designated as beginning wage and/or average wage 
for experiences workers) 
5-information accurate (annual salary or hourly wage) with 

correct benefits statement 
4»information accurate (annual salary or hourly wage) 

without benefits statement 
0-eamings information inaccurate, or earnings information 



293 



CIDS Evaluation, Appendix VII, 19 



not present 

Related occupations, including occupational group 
5«3 or more related occupations and the occupational group 

present and accurate 
4-2 related occupations and the occupational group present 
and accurate 

3»3 or more related occupations present and correct, but 

without the occupational group 
2»2 related occupation, but without occupational group 
l=occupational group present and accurate, but without 

related occupations 
O^related occupations and/or occupational group totally 

inaccurate, or item not present 
Places of employment 
5*major industries and/or types of employers present and 

accurate 

3«partial list or major industries and/or types of employers 

present and accurate 
0«item inaccurate, or not present 
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D. RATING GUIDE FOR SYSTEM REVIEW PANEL RATINGS 



Directions for Rating: The rating of the system will have been carried 
out by the System Review Panel. Use this chart to identify the items* 
which will be weighted twice the others. 



SYSTEM FEATURE EXCELLENT 


GOOD 


FAIR 


POOR ATROCIOUS 


NONE 


1. 


Ease of reading 


5 


4 


3 


2 1 


0 


2. 


Clarity of screen display 


5 


4 


3 


2 1 


0 


3. 


Appeal of the system* 


10 


8 


6 


4 2 


0 


A. 


Relevance of user handbook 


5 


4 


3 


2 1 


0 


5. 


Appropriateness of structured 














search of occupations 


5 


4 


3 


2 1 


0 


6. 


Ability to discontinue search 














& go directly to occ info 


5 


4 


3 


2 ' 1 


0 


7. 


Level of detail of 














occupational information 


5 


4 


3 


2 1 


0 


8. 


Appropriateness and currency 














of terminology 


5 


4 


3 


2 1 


0 


9. 


Clearly defined links between 














occ. and ed. information 


5 


4 


3 


2 1 


0 


10. 


Appropriateness of the 
structured search of post- 














secondary ed. institutions 


5 


4 


3 


2 1 


0 


11. 


Level of detail of college 














and university information 


5 


A 


3 


2 1 


0 
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SYSTEM FEATURE EXCELLENT GOOD FAIR POOR ATROCIOUS NONE 

12. Ability to enter the system at 

points of readiness/interest* 10 8 6 4 2 0 

13. Use in group or classroom 

setting* 10 8 6 4 2 0 

14. Overall ease of use* 10 8 6 4 2 0 

15. Overall rating* 10 8 6 4 2 0 
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